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ABSTRACT 


The  use  of  the  Health  Plan  Employer  Data  and  Information  Set 
(HEDIS)  3.0  provides  the  Military  Health  System  (MHS)  with  a 
platform  to  demonstrate  its  strengths  in  delivering  quality 
managed  care  to  dual-eligible  Medicare/Department  of  Defense 
(DoD)  beneficiaries  and  to  locate  areas  for  improvement.  The 
MHS  is  tasked  with  reporting  22  HEDIS  Medicare  measures  to  the 
Health  Care  Financing  Administration  (HCFA)  to  comply  with  the 
TRICARE  Senior  Prime  (TSP)  demonstration  guidelines.  This  study 
reports  on  the  current  information  model  used  by  the  MHS  to 
collect  one  of  these  measures:  Beta  Blocker  Treatment  After  a 
Heart  Attack.  The  current  information  model  used  by  the  MHS  was 
found  insufficient  to  allow  for  administrative  reporting  of  the 
targeted  measure.  The  study  formulated  an  alternative 
conceptual  information  model  based  upon  the  fundamentals  of 
Total  Data  Quality  Management  (TDQM) .  The  two  models  were 
evaluated  utilizing  Wang's  dimensions  of  information  quality. 

The  alternative  conceptual  model  was  found  to  be  vastly  superior 
to  the  current  information  model  in  every  dimension.  The 
concepts  used  to  formulate  the  conceptual  model,  if  applied, 
would  allow  the  MHS  to  develop  an  effective  information 
management  system  for  the  TSP  demonstration. 
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TRICARE  Senior  Prime  HEDIS  Medicare  Reporting:  Exploring  the 

Information  Quality  Model 

CHAPTER  1 

The  Military  Health  System's  (MHS)  entry  into  the  managed 
care  environment  brings  with  it  many  additional  challenges  and 
requirements.  A  managed  care  system  is  dependent  upon  an 
information  rich  decision  making  environment.  To  achieve  the 
purposed  benefits  of  the  managed  care  environment,  the  health 
service  organizations  must  have  detailed  cost  accounting, 
personnel,  and  clinical  information  systems.  Third  party  payers 
and  more  recently,  individual  consumers  are  requiring  managed 
care  organizations  to  provide  documented  evidence  that  cost 
savings  are  not  resulting  in  a  decreased  quality  of  care 
(Meisenheimer ,  1997) .  As  a  result,  poor  information  management 

practices  have  led  to  devastating  financial  implications  for 
managed  care  organizations  (MCO's)  (Bell,  1998) .  If  a  managed 
care  organization  does  not  have  high  quality  information 
management  systems  and  practices,  it  can  not  survive  in  this 
highly  competitive  and  regulated  industry. 

Conditions  Which  Prompted  the  Study 
The  MHS  Meets  Managed  Care 

The  MHS  currently  serves  approximately  8.2  million  active 
duty  personnel,  eligible  retirees,  and  family  members  at  a  cost 
of  approximately  $15.6  billion-dollars-per-year  (GAO,  1999b). 

The  MHS  faces  the  same  challenges  as  that  of  the  civilian  health 
care  market  place;  namely  to  control  cost,  maintain  adequate 
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access,  and  ensure  quality  of  services.  To  cope  with  the 
reality  of  providing  health  care  in  the  late  20th  century,  the 
MHS  initiated  its  foray  into  managed  care  through  a  vehicle 
entitled  TRICARE  in  1993.  At  the  outset  there  were  concerns  by 
governmental  officials  that  the  MHS  would  not  have  the 
information  management  systems  necessary  to  operate  efficiently 
in  a  managed  care  environment  (GAO,  1995) . 

The  majority  of  information  systems  (IS)  in  the  MHS  at  the 
time  of  the  inception  of  TRICARE,  now  termed  legacy  systems, 
were  the  result  of  years  of  individual  effort  by  services  and 
departments  to  secure  information  valuable  only  to  individual 
functions  (e.g.,  the  Medical  Expense  and  Performance  Reporting 
System  (MEPRS) ) .  Additionally  they  were  designed  to  answer 
questions,  which  were  typical  of  the  retrospective  payment 
health  care  environment.  The  focus  was  on  gross  counts  (e.g., 
the  number  of  bed  days,  the  number  of  outpatient  visits,  etc.) . 
Military  Treatment  Facility  (MTF)  level  funding  was  based  on 
patient  volume,  not  on  how  much  it  cost  to  provide  care  to  an 
individual  patient.  As  such,  the  legacy  systems  where  stove- 
piped,  intended  to  address  a  single  area  of  the  health  care 
enterprise  and  consequently  were  not  designed  to  communicate 
with  each  other. 

In  1995,  the  Assistant  Secretary  of  Defense  (Health  Affairs) 
recognized  the  need  for  an  integrated  high-level  information 
system  to  support  decision-making  efforts  for  TRICARE.  This 
resulted  in  an  MHS  investment  of  over  $300  million  in  the 
Corporate  Executive  Information  System  (CEIS)  (Corey,  Cobler, 


Information  Quality  Model  3 


Haynes,  &  Walker,  1996) .  Since  its  inception,  the  CEIS  program 
has  been  plagued  with  inter-service  rivalry,  project  delays,  and 
cost  over  runs.  At  least  three  distinct  efforts  to  improve  and 
ensure  the  quality  of  information  produced  by  CEIS  were 
initiated  by  the  TRICARE  Management  Activity  (TMA) .  There  is 
also  a  variety  of  individual  efforts  by  the  services,  primarily 
the  Army  and  the  Air  Force,  to  address  information  management 
issues.  There  is  no  data  to  conclusively  verify  that  any  of 
these  costly  efforts  have  resulted  in  sustainable  improvements 
to  the  system.  On  the  contrary,  there  are  volumes  of  evidence 
(GAO,  1999a,  1999b,  1999c,  1999d),  indicating  that  the 
inadequacies  in  the  MHS  data  systems  limit  its  ability  to  manage 
the  delivery  of  health  care  at  both  the  local  and  national 
levels . 

TRICARE  Senior  Prime  Demonstration 

The  Balanced  Budget  Act  of  1997  (BBA)  authorized  the  MHS  to 
conduct  a  3-year  test,  which  it  labeled  Medicare  Subvention;  the 
MHS  prefers  the  term  TRICARE  Senior  Prime  (TSP)  .  The  purpose  of 
this  demonstration  is  to  address  the  concerns  of  the  nearly  1.3 
million  Medicare  eligible  retired  personnel  and  their  families 
over  the  age  of  65.  When  TRICARE  was  initiated  retirees,  upon 
reaching  age  65,  lost  their  right  to  be  enrolled  in  TRICARE 
Prime.  The  retirees  were  forced  to  seek  care  on  a  space- 
available  basis  within  the  MTF,  or  to  seek  care  on  the  civilian 
market  using  Medicare.  Retirees  view  this  as  a  continuation  of 
a  policy  direction  that  began  with  the  elimination  of  Civilian 
Health  and  Medical  Program  of  the  Uniformed  Services  (CHAMPUS) 
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benefits  to  Medicare  eligibles.  Furthermore,  although  not 
specifically  addressed  in  Title  10  of  the  United  States  Code, 
retirees  firmly  believe  they  were  promised  access  to  lifetime 
medical  care  (AFSA,  1998) . 

The  demonstration  project  is  extremely  important  to  MHS 
leaders.  They  view  the  potential  for  Medicare  reimbursement  as 
a  win-win  proposition  for  both  the  MHS  and  retirees.  Retirees 
would  enjoy  the  benefits  of  prime  status  and  would  have  primary 
care  managers,  which  would  ensure  the  continuity  of  care. 
Continuity  of  care  is  extremely  important  when  considering  the 
chronic  nature  of  illness  in  the  elderly  population.  The  MHS 
would  also  be  a  winner  in  that  it  would  first,  maintain  a  bond 
established  with  the  retiree,  and  one  that  many  within  the  MHS 
feel  as  an  "obligation"  to  provide  care.  Secondly,  the  MHS 
would  ensure  its  young  healthcare  providers  have  access  to  the 
geriatric  population,  a  necessary  element  in  the  provision  of  a 
complete  graduate  medical  education.  Thirdly,  the  MHS  would  be 
securing  an  additional  funding  source,  a  factor  of  importance  in 
this  era  of  shrinking  budgets. 

However,  if  the  Medicare  Subvention  demonstration  is  to 
succeed,  then  the  MHS  must  face  the  task  of  meeting  requirements 
set  forth  by  the  Health  Care  Financing  Administration  (HCFA) . 
HCFA,  with  a  few  allowances,  is  treating  the  MHS  as  it  would  any 
other  commercial  managed  care  organization  seeking  to  receive 
Medicare  reimbursement.  There  are  unique  exceptions  enforced  by 
the  BBA .  The  BBA  states  that  the  demonstration  reimbursement 


rates  will  be  95%  of  Medicare+Choice  rates,  adjusted  to  exclude 
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payments  for  direct  and  indirect  medical  education  and 
disproportionate  share  hospitals.  Additionally,  67%  of  the  MHS 
capital  cost  is  excluded  from  the  rate.  The  amount  of 
reimbursement  is  further  constrained  by  the  BBA' s  requirements 
that  the  MHS  ensure  it  is  providing  care  at  a  level  that  is  at 
least  equal  to  the  level  provided  prior  to  the  demonstration, 
termed  baseline  level-of-ef f ort  (LOE) .  HCFA  and  the  MHS  agreed 
upon  a  very  complex  formula  for  establishing  LOE  and  set  the 
base  year  as  1996  and  the  base  LOE  at  $172  million  (GAO,  1999a) . 
Finally,  the  potential  reimbursement  is  capped  at  $50  million  in 
the  first  year,  $60  million  in  the  second  year,  and  $65  million 
in  the  third  year. 

Information  Please 

In  May  1999,  the  United  States  General  Accounting  Office 
(GAO)  published  a  report  entitled  "Medicare  Subvention 
Demonstration:  DoD  Data  Limitations  May  Require  Adjustments  and 
Raise  Broader  Concerns."  The  report  sent  a  clear  signal  to  the 
MHS  that  it  is  in  danger  of  failing  the  demonstration  as  a 
direct  result  of  its  information  management  processes  (GAO, 

1999a) .  The  GAO  report  focuses  on  the  LOE  issue.  It  finds  that 
the  MHS  is  incapable  of  producing  the  necessary  data  to  support 
LOE  determinations.  In  fact,  it  draws  into  question  the  entire 
methodology  for  establishing  the  baseline  estimates.  Perhaps 
even  more  ominous  are  the  inferences  the  GAO  makes  to  the 
broader  concept  of  military  health  care.  As  previously 
mentioned  the  MHS  is  faced  with  competing  in  the  new  era  of 
managed  care.  In  order  to  do  so  it  must  be  able  to  leverage 
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information  to  manage  cost  and  resources  and  maintain  access  to 
and  quality  of  care.  The  GAO  finds  "the  inadequacies  of  DoD's 
data  systems  limit  its  ability,  at  both  the  site  and  national 
levels,  to  manage  the  demonstration  and  deliver  health  care 
[italics  added]"  (GAO,  1999a,  p.  11) .  Furthermore,  "these  data 
problems  also  call  into  question  DoD's  ability  to  manage  its 
overall  health  care  system"  (GAO,  1999a,  p.  16). 

In  response  to  the  GAO  report  the  Office  of  the  Assistant 
Secretary  of  Defense,  Health  Affairs  (OASD (HA) )  took  two 
actions.  First,  it  established  TMA  oversight,  and  instituted  an 
improvement  plan,  for  the  Medical  Expense  and  Performance 
Reporting  System  (MEPRS) .  Secondly,  it  appointed  a  program 
manager  for  data  quality  at  the  TMA  level.  The  Program  Manager 
is  the  head  of  the  Data  Quality  Integrated  Program  Team  (DQ 
IPT ) .  The  stated  purpose  of  the  DQ  IPT  is  to  only  focus  on  data 
quality  issues  related  to  financial,  clinical  workload,  and 
enrollment  data  (GAO,  1999a) .  Obviously,  these  measures  were 
instituted  to  address  specific  concerns  raised  by  the  GAO  in 
their  report . 

HCFA  +  NCQA  =  HEDIS  Medicare 

HCFA  is  treating  the  MHS  like  any  other  commercial  Medicare 
provider  regarding  the  requirement  for  reporting  of  specific 
Health  Plan  Employer  Data  and  Information  Set  (HEDIS)  measures 
related  to  the  Medicare  population.  Assuming  that  the  MHS  was 
able  to  correct  its  financial  data,  clinical  workload  and 
enrollment  data,  and  demonstrate  that  it  exceeds  the  baseline 
LOE,  it  still  may  not  be  able  to  collect  Medicare  reimbursement 
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from  HCF A .  The  reason  is  that  the  MHS  is  currently  unable  to 
comply  with  the  HCFA  requirement  to  report  all  HEDIS  Medicare 
measures  as  set  forth  by  the  National  Committee  for  Quality 
Assurance  (NCQA) (VRI,  1999) . 

The  NCQA  developed  a  set  of  standardized  performance 
measures  (now  in  its  third  iteration--HEDIS  3.0)  to  compare 
health  plan  performance  across  a  wide  range  of  issues. 

Beginning  January  1,  1997,  HCFA,  under  pressure  from  political 

forces,  began  requiring  Medicare  managed  care  plans  to  report 
performance  measures  relevant  to  the  Medicare  population.  The 
reduced  subset  of  the  HEDIS  measures  includes  metrics  that 
appear  closely  aligned  to  evaluate  all  three  areas  of  quality 
(viz.,  structure,  process,  and  outcomes)  as  defined  by 
Donabedian  (1980). 

The  MHS  began  its  first  attempt  to  utilize  HEDIS 
measurements  in  a  non-formal  manner  in  1997.  This  first  attempt 
brought  to  light  many  of  the  failings  of  MHS  information 
systems.  The  TMA  contracted  with  Vector  Research,  Inc.  (VRI)  to 
evaluate  the  feasibility  of  data  collection  and  to  collect 
available  and  appropriate  measures  for  FY  96  data  (VRI,  1997) . 
The  TMA  requested  that  VRI  breakout  the  measures  into  two 
separate  report  groupings:  active  duty  (AD)  and  non-active  duty 
(NAD) .  For  the  purposes  of  this  discussion,  the  focus  will  only 
be  on  the  NAD  reports  since  they  most  closely  associate  with  the 
TSP  population.  VRI  conducted  a  similar  analysis  in  1998  for 
the  TMA  using  FY97  data  (VRI,  1998a) .  VRI  has  provided  the  TMA 
with  a  functional  analysis  on  the  feasibility  of  collecting 
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Table  1 

Comparison  of  MHS  HEDIS  Reporting  Efforts  1997  -  1999. 


Report /Year 

NAD  FY  9  6a 

NAD  FY 97b  TSP 

CY98c 

Total  number  of  potential  HEDIS 
Measures 

71 

56 

33 

Measures  Included  in  Report/Year 

56 

27 

12 

Number  of  Measures  which  met  all 
HEDIS  Technical  Specifications 

15 

14 

9 

Note .  NAD  =  Non-Active  Duty.  TSP  =  TRICARE  Senior  Prime. 

aData  in  column  are  derived  from  Military  Health  System  FY96 

HEDIS  3.0  Report  for  Nonactive  Duty  TRICARE  Prime  Enrollees  by 

Vector  Research,  Inc.,  1997,  Arlington,  VA:  Author. 

bData  in  column  are  derived  from  Military  Health  System  FY97 

HEDIS  3.0  Report  for  Nonactive  Duty  TRICARE  Prime  Enrollees  by 

Vector  Research,  Inc.,  1998,  Arlington,  VA:  Author 

cData  in  column  are  derived  from  Fiscal  Year  1997  Health  Employer 

Data  and  Information  Set  (HEDIS)  3.0/1998  Baseline  Report  for 

TRICARE  Senior  Prime  Enrollees  at  Madigan  Army  Medical  Center  by 

Vector  Research,  Inc.,  1998,  Arlington,  VA:  Author. 

HEDIS  Medicare  measure  for  the  TSP  population  (VRI,  1999) .  Table 
1  highlights  the  difficulty  the  MHS  has  had  in  collecting  HEDIS 
measures . 

The  NCQA  continuous  to  refine  the  HEDIS  measurement  system. 
Thus,  the  NCQA  revises  the  number  of  measures  from  year  to  year. 
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and  makes  significant  methodological  changes  in  already  existing 
measures.  Therefore,  direct  comparison  for  HEDIS  measures  from 
one  year  to  the  next  is  not  always  appropriate.  The  important 
concept  to  observe  is  the  MHS'  significant  inability  to  report 
the  measures  in  accordance  with  the  NCQA's  technical 
specifications . 

The  MHS  was  under  no  direct  financial  pressure  during  FY96 
and  FY97  to  report  any  HEDIS  measure.  Therefore,  it  exercised 
great  discretion  in  picking  measures,  which  it  considered 
beneficial  to  the  overall  quality  initiatives  within  the  MHS. 
Additionally,  at  that  time  there  was  not  a  third  party  that 
mandated  compliance  with  the  NCQA  HEDIS  measurement  methodology, 
allowing  the  MHS  to  utilize  "HEDIS-Like  Measures"  [TMA 
terminology] (TMA,  1999) . 

The  TMA,  being  unable  to  conform  to  the  NCQA  technical 
specifications  for  the  measures,  utilized  HEDIS-Like  measures  to 
address  the  spirit  of  the  measure.  This  suggests  that  the  TMA 
considered  the  measure  to  be  highly  valuable,  however,  available 
information  practices  did  not  allow  for  accurate  reporting.  The 
TMA  opted  to  report  50%  (36)  of  the  HEDIS  measures  in  FY96. 

Only  42%  (15)  of  these  met  HEDIS  technical  specifications.  The 
TMA  chose  to  report  48%  (27)  of  the  available  HEDIS  measures  in 
FY97.  During  this  reporting  period,  52%  (14)  met  the 

specifications.  This  indicates  that  information  management 
practices  deprived  the  TMA  of  accurately  reporting  52%  and  48% 
respectively  of  the  measures  that  the  MHS  deemed  important  to 
measuring  quality  within  its  health  care  system. 
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HCFA  now  provides  a  third-party  control  for  HEDIS 
measurement  and  removes  the  latitude  formally  employed  by  the 
TMA .  Thus,  for  the  TSP  demonstration  HEDIS-Like  measures  are 
not  an  option.  The  first  study  devoted  to  analyzing  HEDIS 
measurement  for  the  TSP  demonstration  found  that  only  27%  (9  of 
33)  of  the  measures  met  the  technical  requirements  of  the  NCQA 
(VRI,  1998b) .  The  number  of  HEDIS  Medicare  measures  the  MHS 
will  be  required  to  report  on  in  2000  is  twenty-two  (see  Table  3 
on  page  45) .  Arguably,  the  MHS  has  not  made  any  significant 
improvements  over  the  last  three  years  in  its  information 
management  practices,  which  would  allow  for  the  accurate 
reporting  of  all  required  HEDIS  Medicare  measures. 

Statement  of  the  Problem  or  Question 

The  quality  of  data  processes  in  the  MHS  does  not  support 
and,  in  fact,  hinders  business  and  clinical  operations.  The 
problem  is  obvious  to  the  GAO,  HCFA,  the  NCQA,  and  even  the  MHS. 
The  MHS  is  faced  with  the  task  of  reporting  HEDIS  Medicare 
measures  in  accordance  with  NCQA  and  HCFA  guidelines.  Currently 
the  data  processes  do  not  allow  for  complete  and  accurate 
reporting  of  all  measures.  This  study  seeks  to  define  a  method 
by  which  the  MHS  can  improve  its  information  management 
practices  to  support  the  TSP  demonstration. 

Literature  Review 

The  purposes  of  this  literature  review  are  to  provide  the 


reader  with  an  appreciation  for  the  information  management 
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problem  not  only  in  the  MHS  but  also  in  the  business  world  at 
large.  It  will  develop  the  operational  definitions  for  data 
quality  and  information  management,  which  as  will  be  shown  are 
varied.  Finally,  it  will  provide  potential  models  for 
evaluating  the  TSP  HEDIS  measurement  problem  facing  the  MHS. 

The  Cost  of  Poor  Data  Quality 

Napoleon  Bonaparte  is  quoted  as  saying:  "War  is  ninety 
percent  information."  The  war  on  crime  is  not  going  well  if  one 
considers  that  50%  to  80%  of  the  computerized  criminal  records 
in  the  United  States  contain  incomplete,  inaccurate,  or 
ambiguous  data  (Strong,  Lee,  &  Wang,  1997) .  This  is  the 
information  age.  Governments,  businesses,  and  individuals 
require  computer  generated  data.  These  data  form  information 
that  drives  political,  business,  and  personal  decision  making. 
What  is  the  cost  if  the  decision  making  process  is  built  upon 
the  foundation  of  faulty  data?  Strong  et  al .  (1997)  estimated 

that  the  impact  to  society  is  billions  of  dollars.  Redman 
(1998)  stated  that  a  typical  mid  to  large  size  corporation 
looses  8%  to  12%  of  its  revenues  to  poor  data  quality  and  for  a 
service  organization  40%  to  60%  of  expenses  are  related  to  poor 
data  quality. 

These  costs  are  both  direct  and  indirect.  The  direct  costs 
are  primarily  related  to  the  expense  that  companies  incur  trying 
to  clean-up  data  already  in  their  information  systems.  The 
indirect  costs  are  derived  from  the  impact  the  faulty 
information  and  information  systems  have  on  employee 
satisfaction  and  productivity,  customer  satisfaction,  and 
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corporate  strategic,  tactical,  and  operational  decision  making. 
The  indirect  cost  of  poor  quality  data  for  the  MHS  on  just  the 
TSP  project  alone  could  total  $175  million  over  the  next  three 
years  (GAO,  1999a). 

The  study  will  assess  the  direct  costs  first.  Hankins 
(1999)  estimated  the  market  for  data  cleansing  efforts  will  be 
more  than  $300  million  a  year  by  2000.  The  costs  were 
accelerating  as  more  companies  turn  their  efforts  to  data 
warehousing  efforts.  Data  warehouses  are  designed  to  take 
subsets  of  data  from  a  company' s  diverse  legacy  information 
systems  (viz.,  accounting,  marketing,  personnel,  and  inventory) 
and  combine  them  into  a  central  repository.  The  data  warehouse 
forms  a  centralized  source  from  which  the  company  is  able  to 
"mine"  information  for  corporate  decision  making. 

It  is  a  given  that  each  IS  will  contain  some  level  of  faulty 
data.  On  an  individual  level,  users  of  the  systems  have 
intimate  knowledge  of  the  data  and  processes  feeding  the  systems 
and  internalize  known  faults  into  their  decision  making 
processes  (Ballou  &  Tayi,  1999)  .  This  level  of  understanding  is 
lost  to  the  users  of  the  data  warehouse.  Therefore,  either  they 
will  take  the  information  provided  by  mining  at  face  value  or 
they  will  discount  obviously  faulty  information  and  loose 
confidence  in  any  information  provided  by  the  warehouse. 

Celko  &  McDonald  (1995)  were  one  of  the  first  to  comment  on 
the  havoc  that  poor  data  quality  would  have  on  data  warehouses. 
Poor  data  qualities  in  legacy  systems  have  an  exponential 
compounding  effect  on  data  warehouses.  The  authors  found  that 
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80%  of  the  queries  in  a  data  warehouse  will  access  the  small 
area  of  the  database  with  the  most  ailments.  They  firmly  stated 
that  companies  can  not  ignore  or  trivialize  problems  with  the 
legacy  data.  If  companies  fail  to  correct  legacy  data,  the 
oversight  will  brutally  assert  itself  inside  the  data  warehouse. 
They  warned  businesses  not  to  assume  that  data  is  correctable 
once  warehoused.  Imagine  the  dismay  of  an  executive  or 
government  official  to  find  that  a  hundred  million  dollar 
project  is  useless  unless  they  spend  millions  more  to  correct 
data  quality  problems.  Redman  (1996)  estimated  50%  of  the  cost 
incurred  in  implementing  a  data  warehouse  is  directly  attributed 
to  cleaning  up  faulty  source  data. 

There  are  at  least  three  reasons  why  executives  are  facing 
this  dilemma  in  relation  to  their  data  warehouse  projects.  The 
first  is  because  organizations  have  focused  on  quantity  versus 
quality  (Hankins,  1999) .  The  second,  is  the  lack  of  common  data 
definitions  across  the  system  (Hemblen,  1998) .  The  third  is 
attempting  to  use  data  for  purposes  outside  the  scope  of  its 
original  collection  intent  (Hannan,  Racz,  Jolis,  &  Peterson, 

1997) .  The  second  and  third  problems  are  more  germane  to  the 
warehouse  problem.  The  quantity  over  quality  problem  is  a  root 
cause,  which  has  festered  to  painfully  manifest  itself  in  the 
data  warehouse. 

The  problem  of  the  lack  of  common  definitions  has  deep 
political  roots.  As  Burzynski  (1998)  indicates,  the  MHS  is 
formed  from  three  services  with  different  cultures,  regulations, 
management  priorities,  and  policies  governing  the  use  of 
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information  systems.  The  implications  for  the  MHS  are  readily 
apparent  to  the  casual  observer.  For  example,  three  years  after 
implementing  Ambulatory  Procedure  Visits  in  the  MHS,  there  is 
still  not  a  standard  definition  across  the  services  for  its  use. 
The  same  is  true  for  counting  bassinet  (newborn)  bed  days.  Even 
the  most  basic  items  are  fraught  with  stagnating  political 
implications,  such  as  the  definition  of  a  provider.  Hemblen 
(1998)  believes  that  the  $800  million  spent  on  the  CEIS  project 
could  be  wasted  if  the  services  fail  to  achieve  standardized 
data  definitions. 

A  more  insidious  data  quality  problem  is  the  attempt  to  use 
data  for  other  than  their  primary  intent.  Hannan  et.al  (1997) 
found  that  Medicare  claims  data  found  in  the  typical  hospital 
administrative  database  is  a  poor  source  for  the  evaluation  of 
the  effectiveness  of  care.  The  reasons  are  not  always  obvious 
to  the  uninformed. 

The  administrative  databases  in  our  health  care  systems  were 
designed  to  collect  encounter  data  for  reimbursement  purposes. 
The  most  prevalent  data  model  is  based  upon  a  financial 
framework  developed  by  Joanne  Finely  and  a  group  of  Yale 
researchers  (Ulman  &  Kominski,  1984) .  The  purpose  of  this  data 
model  is  to  apportion  hospital  admissions  into  a  set  of  mutually 
exclusive  categories.  The  data  model  is  not  based  upon  a 
clinical  framework,  thus  it  fails  to  capture  the  elements 
important  to  determine  effectiveness  of  care.  Never  the  less, 
there  are  many  attempts  to  use  this  data  model  to  make  quality 
of  care  determinations.  The  importance  of  the  appropriate  data 
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model  can  not  be  over  emphasized.  For  example,  the  essence  of 
the  Y2K  problem  was  the  result  of  a  poorly  designed  data  model. 

Arguably,  the  root  of  the  problem  is  that  organizations 
focus  too  much  on  quantity  and  not  enough  on  quality. 
Computerization  has  made  it  very  easy  for  organizations  to 
collect  mass  quantities  of  data,  especially  in  health  care 
organizations.  However,  these  data  were  collected  as  a  by¬ 
product  of  some  core  business  processes  and  no  effort  was  made 
to  treat  data  as  products  in  their  own  right  (Levitin  &  Redman, 
1998)  . 

Consider  the  difference  between  a  checkbook  and  an 
accountant's  ledger  (see  Figure  1) .  A  person  will  record 
transactions  in  a  personal  checkbook  as  a  by-product  of  normal 
everyday  activities  (e.g.  paying  the  electric  bill) .  The  act  of 
recording  and  capturing  the  data  is  not  the  focus  or  primary 
objective  of  the  person.  Most  organizations  have  collected 
their  data  in  much  the  same  way,  and  as  a  result  they  have  as 
much  difficulty  reconciling  the  data  as  people  have  balancing 
personal  checkbooks.  Compare  this  to  an  accountant's  ledger,  in 
which  every  entry  is  a  core  activity.  The  accountant  follows  a 
very  well  defined  and  prescriptive  process  for  creating, 
entering,  and  capturing  the  data.  There  are  numerous  quality 
control  mechanisms  built  into  the  process.  Therefore,  it  is  not 
surprising  to  find  that  the  application  of  the  personal 
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Figure  1.  Checkbook  vs .  Ledger 

checkbook  approach,  or  by-product  approach,  to  data  collection 
has  resulted  in  field  level  error  rates  ranging  between  1%  and 
75%  (Redman,  1996). 

Some  processes  in  the  health  care  environment  have  well 
defined  processes  for  collecting  the  data,  however  quality 
control  practices  are  either  not  implemented  or  not  followed. 
Since  the  inception  of  the  Prospective  Payment  System,  hospitals 
have  hired  thousands  of  medical  record  encoders  to  review 
medical  records  and  convert  the  information  to  Diagnosis-Related 
Groups  (DRG)  and  Current  Procedural  Terminology  (CPT)  codes. 
Laing  (1992)  was  one  of  the  first  to  formally  study  and  find 
that  the  quality  of  the  coding  process  is  in  question.  Ackland 
&  Chandraraj  (1997)  found  in  a  review  of  coded  emergency  room 
records  that  there  was  a  73%  error  rate  in  any  given  field,  a 
61%  error  in  diagnosis  coding  and  a  45%  error  rate  in  procedure 
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coding.  This  was  a  foreign  study  conducted  in  Australia. 
However,  the  results  mirror  those  found  by  Facisweski,  Broste,  & 
Fardon  (1997)  in  their  review  of  spinal  surgery  coding.  These 
authors  found  a  72%  error  rate  in  coding  of  specific  diagnosis 
among  six  U.S.  hospitals.  Facisweski  et  al . ,  additionally 
highlight  the  great  variation  in  the  coding  processes,  both 
intra  and  inter-hospital. 

Quality  problems  are  not  confined  to  encoding  efforts.  They 
are  also  evident  in  every  other  information  system  the  typical 
health  care  enterprise  employs.  Pumphrey,  Fuller,  Radosta,  & 
Dittrich  (1999)  found  that  35%  of  the  entries  in  a  utilization 
management  database  used  to  record  the  status  of  patients  were 
in  error.  The  problem  was  caused  by  the  utilization  managers' 
reliance  upon  others  to  update  the  information  as  a  by-product 
of  their  health  care  processes.  The  implications  are  staggering, 
imagine  an  administrator  not  being  able  to  accurately  identify 
the  location  of  35%  of  his/her  patients!  The  hospital  promptly 
identified  a  formal  process  for  collecting  and  ensuring  the 
quality  of  the  data. 

Poor  data  management  practices  have  cost  large  managed  care 
companies  millions  of  dollars.  Peter  Kongstvedt,  a  leading 
authority  on  the  managed  care  industry,  has  stated  that  the  data 
quality  problem  is  severe  among  managed  care  organizations  (As 
reported  in  Bell,  1998) .  The  cost  of  poor  data  management 
practices  is  also  effecting  those  in  the  government  sector.  The 
Department  of  Veterans  Affairs  concern  for  the  quality  of  their 
data  prompted  them  to  allocate  $1  million  dollars  in  1999  to 
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correct  the  coding  of  diagnosis  and  medical  services  ("VA 
initiative,"  1999).  Additionally,  the  problem  is  effecting 
those  who  would  wish  to  monitor  the  health  care  industry.  The 
NCQA  was  faced  with  the  dilemma  of  corrupt  data  sets,  giving 
many  managed  care  firms  a  convenient  excuse  to  drop  optional 
HEDIS  reporting  (Bell,  1998) .  The  NCQA  subsequently  adopted  a 
policy  of  having  the  MCO's  pay  for  an  audit  of  their  data, 
information  systems,  and  information  management  practices  as  a 
precondition  to  submitting  data  for  HEDIS  measurement  (NCQA, 

1999)  . 

Now  that  some  of  the  direct  costs  of  poor  information 
management  practices  have  been  explored,  attention  is  directed 
toward  the  indirect  costs.  There  are  no  simple  methodologies 
for  applying  specific  dollar  amounts  to  these  costs.  However, 
logical  induction  gives  that  if  information  management  practices 
produce  data,  which  yield  information,  which  in  turn  guides 
decision  making;  then  deficient  information  management  practices 
produce  faulty  data,  erroneous  information,  and  ultimately 
flawed  or  at  least  impaired  decision  making.  The  literature 
would  appear  to  support  this  assumption.  Redman  (1998)  stated 
that  many  of  the  problems  facing  today's  executives  have  poor 
data  quality  practices  at  their  roots. 

The  indirect  cost  of  poor  information  management  practices 
is  levied  at  the  strategic,  tactical,  and  operational  levels 
within  an  organization.  Peter  Drucker  stated,  "The  organization 
of  the  future  is  rapidly  becoming  a  reality — a  structure  in 
which  information  serves  as  the  axis  and  as  the  central  support 
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system."  Ginter,  Swayne,  and  Duncan  (1998)  believe  that 
information  management  is  of  equal  importance  in  the  modern 
health  care  organization  as  such  traditional  functions  of 
marketing,  human  resources,  and  finance.  If  information 
management  is  of  equal  importance  to  the  strategic  health  of  an 
organization  one  wonders  why  32%  of  health  care  executives 
encounter  errors  in  their  information  systems  all,  or  most  of, 
the  time  (Bean,  1994) .  If  an  equal  number  of  executives  faced 
errors  in  their  financial  reports,  one  wonders  how  long  they 
would  remain  executives.  Never  the  less,  the  inadequacies  of 
information  management  makes  it  extremely  difficult  for 
organizational  leaders  to  set  and  execute  a  strategy,  or  to 
align  the  organization  (Redman,  1996). 

Tactical  decision  making  is  equally  affected  by  poor  data 
management  practices.  Questions  such  as  how  much  to  spend,  or 
how  to  allocate  resources,  are  clouded  by  the  lack  of  reliable 
information.  In  the  typical  MTF,  allocation  of  resources  is 
driven  more  by  gut-level  decision  making  than  quantitative 
analysis  of  the  problem.  Not  because  the  leaders  lack  the 
knowledge  or  desire  to  use  such  tools,  but  because  access  to 
believable  data  is  not  forthcoming  from  current  information 
systems.  Instinctively  the  service  leaders  believed  the  MHS 
could  profit  from,  or  at  least  break  even  on,  the  TSP 
initiative.  However,  leadership  could  not  identify  the  cost  of 
the  current  level  of  effort  . 

At  the  operational  level,  employees  and  customers  are 
frustrated  by  the  very  systems  designed  to  enhance  their 
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interactions.  Poor  information  practices  are  a  huge  employee 
and  customer  dissatisfier  (Redman,  1996) .  The  deployment  of  the 
Ambulatory  Data  System  (ADS)  in  the  MHS  provides  an  excellent 
example.  The  ADS  was  deployed  without  consideration  of  the 
current  business  practices  of  the  typical  MTF  clinic.  Further, 
the  system  was  not  fully  tested  before  deployment  leading  to  IS 
and  information  management  failures,  which  resulted  in  employee 
frustration  and  consternation  (OASD (HA) ,  1997). 

Haung,  Lee,  and  Wang  (1999)  find  that  quality  information 
leads  to  improved  customer  service,  customer  satisfaction,  and  a 
stronger  customer  relationship.  The  MHS  customers  are 
frustrated  by  the  information  management  systems.  This  author's 
own  experience  illustrates  this  point  .  Upon  presenting  to  the 
dental  clinic  for  a  routine  examination,  the  author  found  that 
he  was  "not  in  the  system."  However,  the  author  had  watched  the 
clerk  only  one  week  before  dutifully  going  through  the  five 
minute  booking  process  on  her  terminal.  She  even  supplied  him 
with  written  verification  of  the  appointment.  Never  the  less, 
on  the  day  of  the  expected  interaction  no  record  of  the 
appointment  was  in  the  system,  resulting  in  another  booking 
procedure,  a  delay  in  treatment,  and  another  day  spent  visiting 
the  dental  clinic.  Poor  information  management  adversely 
affects  the  most  fundamental  business  practices  in  a  health  care 
organization  . 

In  concluding  this  section,  the  "dark  side"  to  the 
information  age  was  explored.  Insufficient  data  quality 
practices  are  rampant  through  out  the  business  world,  and  are 
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not  confined  to  the  health  care  sector  or  even  to  the  MHS  in 
particular.  The  quality  of  the  data  poses  special  problems  in 
the  health  care  industry  because  of  the  sheer  volume  of  data 
that  are  collected.  It  also  poses  special  problems  in  the  MHS 
due  to  its  political/organizational  configuration.  The 
literature  appears  to  support  the  assumption  that  poor  data 
quality  practices  have  both  direct  and  indirect  costs  to  the 
organization.  Poor  information  management  has  a  cost  and,  in 
fact,  is  costing  health  care  organizations  millions  of  dollars. 
Poor  data  quality  practices  attacks  the  organization  at  the 
strategic,  tactical,  and  operational  levels.  Finally,  one  is 
strongly  encouraged  to  agree  with  Huang  et .  al  (1999)  that 
organizations  are  finding  quality  information  as  their  most 
valuable  resource. 

Defining  Information  Quality 

The  distinction  between  data  quality  and  information  quality 
is  often  times  blurred  and  used  interchangeably  in  the 
literature.  This  author  will  attempt  to  make  a  clear 
distinction  for  the  purposes  of  this  study.  However,  as  these 
two  examples  from  Webster's  New  World  Dictionary  (1990) 
illustrate,  the  task  is  daunting.  Webster's  defines  data  as 
"facts  or  figures  from  which  conclusions  can  be  drawn."  The 
definition  of  information  is  "something  told  or  facts  learned; 
data  stored  in  or  retrieved  from  a  computer." 

The  common  definition  used  among  those  responsible  for  data 
quality  in  the  MHS  comes  from  American  National  Dictionary  for 
Information  Systems  (1991),  which  states  data  quality  is  "the 
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correctness,  timeliness,  accuracy,  completeness,  relevance,  and 
accessibility  that  makes  data  appropriate  for  use"  (cited  in 
Burzynski,  1998) . 

A  common  paradigm  used  to  describe  the  relationship  between 
data  and  information  is:  data  information  knowledge 
(Dertouzos,  1997) .  From  this  releat ionship  one  can  descern  that 
data  are  a  well  from  which  information  may  be  drawn.  However, 
one  could  hypothesize  that  since  there  is  a  process  involved  in 
assembling  data  into  information,  quality  data  does  not 
necessarily  yield  quality  information.  The  research  literature 
and  statistical  texts  are  filled  with  examples  of  high  quality 
data  poorly  presented  or  subjected  to  inappropriate  analysis  and 
thus  leading  to  information  of  inferior  quality.  Therefore, 
focusing  only  upon  the  data  will  not  improve  the  business 
organization.  Attention  must  also  be  given  to  the  processes 
that  turn  the  data  into  information. 

Defining  data  quality  is  the  first  step  in  defining 
information  quality.  Tayi  and  Ballou  (1998)  define  data  quality 
as  "fitness  for  use",  which  suggests  that  the  concept  of  data 
quality  is  relative.  This  concept  of  relativity  is  also  found 
in  Redman's  (1996)  definition  of  data  quality:  "A  datum  or 
collection  of  data  X  is  of  higher  (or  better)  quality  than  a 
datum  or  collection  of  data  Y  if  X  meets  customer  needs  better 
than  Y"  (p .  19). 

These  definitions  present  little  practical  appeal,  because 
they  offer  no  absolutes.  These  researchers  imply  that  there  is 


no  authorative  source  which  states  whether  a  datum  is  correct  or 
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incorrect.  The  answer  is  akin  to  popular  art  where  the 
appraisal  is  in  the  eye  of  the  beholder. 

This  author  finds  the  above  definition  of  data  quality 
insufficient  and  will  provide  a  contrapuntal  definition.  A 
computer  is  actually  a  simple  machine.  It  can  only  understand 
and  process  a  binary  set  of  numbers  containing  0's  and  l's. 

Thus  the  smallest  datum  in  our  information  systems  is  called  a 
bit.  Dertouzos  (1997)  stated,  "Bits  maybe  as  plentiful  as  the 
sand,  but,  like  the  sand  they  are  useless  unless  fashioned"  (p. 
234) .  This  fashioning  of  bits  is  based  upon  a  model  of  the 
information  the  user  wishes  to  capture. 

For  example,  consider  a  person  wishing  to  use  a  digital 
camara  to  place  a  picture  in  this  text.  The  first  step  would  be 
to  define  the  requirements.  For  this  contrived  example,  the 
requirement  will  be  a  head  and  shoulders  portrait  of  the  author. 
The  requirements  should  lead  naturally  to  a  definition  of  the 
data  model  which  will  ultimately  capture  the  data.  If  the 
digital  camera  was  pointed  at  a  dog  and  picture  taken,  the 
camera  would  faithfully  take  the  light  and  covert  it  to  binary 
data  in  the  form  of  O's  and  l's.  This  data  could  then  be 
reassembled  by  a  computer  to  produce  a  picture  of  not  the  author 
but  a  dog  (some  people  may  know  the  difference) !  Are  the  data 
wrong?  No,  as  long  as  a  visible  picture  came  out  the  data  are 
not  corrupted.  The  problem  is  that  the  information  model  is 
incorrect  based  on  the  information  requirements.  What  would 
happen  if  the  operator  failed  to  upload  the  image  to  a  personal 
computer  before  trying  to  access  it?  Is  the  data  quality  bad? 
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No,  the  0's  and  l's  are  still  faithfully  stored  in  the  digital 
camera  in  the  proper  sequence.  The  problem  is  an  information 
chain  and  access  problem.  The  operator  failed  to  carry  out  a 
required  sequence  in  the  information  chain,  thus  access  to  the 
image  is  denied.  This  again  is  an  information  quality  problem. 
One  final  scenerio:  what  happens  if  the  operator  took  a  head  and 
shoulders  portrait  of  the  author,  uploaded  it  to  the  computer, 
but  the  computer  rearranged  the  binary  sequence  to  make  the 
image  unrecognizable.  In  this  instance,  the  malfunction  is  a 
data  quality  problem  because  the  integrity  of  the  source  data  is 
defiled  (Huang  et .  al,  1999) . 

Therefore,  it  is  agruable  that  if  the  term  data  quality  is 
used  to  denote  the  integrity  of  the  binary  0's  and  l's  in  the 
computer  system,  then  there  is  a  source  of  truth.  It  may  also 
be  surmized  that  data  quality  does  not  infer  information 
quality.  The  two  are  distinct,  abliet  related,  concepts  which 
should  not  be  confused. 

Dr.  R.  Wang  and  associates  from  the  Total  Data  Quality 
Management  program  at  the  Massachusetts  Institute  of  Technology 
conducted  extensive  ontological  research  into  the  term 
information  quality.  They  hypothesize  that  the  fundamental  role 
of  the  information  system  is  to  provide  an  accurate 
representation  of  a  real-world  system  as  perceived  by  the  user 
(Huang  et .  al,  1999) .  Through  extensive  research  efforts, 
they  have  deterimed  that  information  quality  has  four  domains 
(or  categories)  and  sixteen  dimensions  as  displayed  in  Table  2. 
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Table  2 

Information  Quality  Categories  and  Dimensions 


IQ  Categories 

IQ  Dimensions 

Intrinsic  IQ 

Accuracy,  objectivity,  believability , 

reputation 

Contextual  IQ 

Relevancy,  value-added,  timeliness, 

completeness,  amount  of  information 

Representational  IQ 

Interpretability ,  ease  of  understanding, 

concise  representation,  consistent 

representation 

Accessibility  IQ 

Access,  security 

Note :  From  "A  product  perspective  on  total  data  quality 
management"  by  R.Y.  Wang,  1998,  Communications  of  the  ACM, 
41(2),  p.  60.  Copyright  1998  by  ACM,  Inc.  Reprinted  by 
permission . 


From  Dr.  Wang's  analysis,  there  is  only  one  dimension  of 
information  quality  that  fits  the  author's  definition  of  data 
quality:  accuracy.  Therefore,  when  Dr.  Wang  defines  information 
quality  as  "fitness  for  use  by  information  consumers."  The 
relative  nature  that  the  previous  authors  attributed  to  data 
quality  is  actually  an  attribute  of  information  quality. 
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There  is  probably  some  question  as  to  why  the  author  is 
emphasizing  the  contrast  between  data  quality  and  information 
quality.  The  answer  will  become  more  apparent  when  solutions 
for  the  information  quality  problem  are  explored  and  especially 
when  the  collection  of  TSP  HEDIS  Medicare  measures  is  addressed. 
For  now  the  conceptual  difference  is  best  explained  by  using  an 
allegory  from  Steven  Covey. 

Dr.  Covey  provides  the  example  of  the  difference  between  a 
manager  and  a  leader.  To  paraphrase  Dr.  Covey,  a  good  manager 
of  a  group  of  loggers  ensures  that  each  worker  is  maximizing 
production.  The  leader  on  the  other  hand  is  climbing  a  tree  to 
make  sure  the  loggers  are  in  the  right  forest.  An  equal 
comparison  exists  between  improving  data  quality  and  information 
quality . 

Data  quality  improvements  are  extremely  expensive  and  time 
consuming.  In  fact,  Ballue  and  Tayi  (1999)  have  developed  a 
utility  model  to  help  organizations  quantitatively  determine 
where  they  could  get  the  most  bang  for  their  data  quality  buck. 
Data  quality  efforts  in  systems  as  large  and  complex  as  the  MHS, 
where  you  have  literally  thousands  of  transfer  points,  are 
needed.  However,  they  are  not  the  best  place  to  start. 

The  author  believes  that  before  the  organization  even 
considers  attempting  to  correct  the  data  in  its  legacy  systems, 
it  should  re-evaluate  its  information  needs.  The  MHS  is 
spending  thousands  of  dollars  to  increase  the  quality  of  its 
data.  Unfortunately,  even  if  the  data  was  100%  accurate  and 
complete  it  still  would  not  provide  the  information  necessary  to 
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answer  basic  business  questions,  simply  because  the  legacy  MHS 
information  model  does  not  fit  the  TRICARE  managed  care 
information  model.  In  short,  they  have  a  picture  the  dog  from 
the  earlier  illustration. 

The  legacy  MHS  information  model  is  based  on  information  as 
a  by-product,  the  maximization  of  freedom  of  choice  by  each  of 
the  services,  and  counting  widgets--the  former  basis  of  budget 
allocation.  The  TRICARE  managed  care  information  model  demands 
that  information  be  treated  as  a  core  business  process,  the 
maximization  of  standardization  among  the  three  services,  and 
the  documentation  of  financial  cost  and  clinical  outcome  for 
each  individual  episode  of  care.  A  further  discussion  of  models 
will  be  provided  in  a  later  section. 

In  summary,  data  quality  refers  to  the  integrity  of  binary 
bits.  Data  quality  has  an  absolute  reference  point--did  the 
system  record,  store,  process,  and  transmit  all  bits  of 
information  in  the  correct  sequence.  Data  quality  is  necessary, 
but  not  sufficient,  for  information  quality.  Information 
quality  is  a  relative  term  based  upon  the  consumer' s 
determination  of  fitness  for  use.  Information  quality  consists 
of  four  domains  and  sixteen  categories. 

Solutions  for  Information  Quality 

The  literature  provides  a  rich  source  of  information  on 
theory  and  practice  in  solving  the  information  quality  dilemma 
facing  the  MHS.  We  will  begin  our  review  with  current  theory  to 
solving  these  issues  and  then  move  into  a  further  discussion  of 
the  delineation  between  data  quality  and  information  quality. 
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Information  Management  Quality  Theory 


All  of  the  current  Information  Management  Quality  (IMQ) 
theory  is  based  on  the  foundation  provided  by  Deming.  Deming 
provides  the  conceptual  design  for  improving  quality  within  any 
organization  (Walton,  1986) .  Deming' s  concepts  are  widely 
taught  so  they  will  not  be  detailed  here.  At  the  heart  of 
Deming' s  theory  are  the  Plan-Do-Check-Act  (PDCA)  cycle,  process 
control,  and  continuous  quality  improvement.  These  concepts  are 
modified  and  used  repeatedly  in  various  forms  by  a  number  of 
authors.  The  application  of  Deming  to  the  information 
management  problem  eventually  takes  on  the  form  of  Total  Data 
Quality  Management  (TDQM)  (Buryznski,  1998;  Corey  et .  al,  1996; 
Huang  et .  al,  1999;  Wang,  1998). 

The  most  basic  model  is  that  supplied  by  the  Department  of 
Defense  (DoD)  (Buryznski,  1998;  Corey  et .  al,  1996) .  The  DoD 
promulgates  its  model  through  the  Department  of  Defense  (DoD) 
Data  Quality  Management  Guidelines .  The  steps  in  the  DoD 
process  are  depicted  in  Figure  2. 

There  is  some  evidence  to  support  the  notion  that  the  MHS 
has  established  a  TDQM  environment,  at  least  at  the  upper 
echelons.  Establishing  a  TDQM  environment  entails  establishing 
a  culture  that  regards  data  quality  as  being  important.  Recent 
GAO  reports  (GAO,  1999a,  1999b,  1999c,  1999d)  and  the  TSP 
demonstration  have  certainly  provided  the  MHS  with  an  incentive 
to  increase  data  quality.  Many  if  not  all  of  the  service 
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Evaluate  TDQM 
Process 


Establish  TDQM  Environment 


DOD 

TDQM 

Process 


Scope  Data  Quality  Project 
and  Develop  Implementation 
Plan 


Implement  Data  Quality  Project 


Figure  2.  DoD  TDQM  Process 

Surgeons  General  has  spoken  of  the  dire  need  to  address  this 
issue.  The  Assistant  Secretary  of  Defense,  Health  Affairs,  has 
issued  memorandum  on  the  importance  of  data  quality  in  the  MHS . 
There  is  certainly  a  great  deal  of  money  and  manpower  devoted  by 
the  TMA  to  address  the  data  quality  problem.  However,  whether 
all  of  this  top-level  focus  is  filtering  down  to  the  MTF  level 
is  not  so  easily  discernable. 

The  need  for  top  level  involvement  is  a  requirement  for  any 
successful  data  quality  project.  The  concept  of  improving  data 
quality  means  that  there  is  a  state  of  poor  data  quality.  To 
move  from  the  state  of  poor  data  quality  to  a  state  of  good  data 
quality,  entails  that  a  change  in  processes  must  occur. 
Leadership  is  a  fundamental  element  of  change.  According  to 
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Redman  (1996),  leaders  are  tasked  with  five  responsibilities. 

The  first  is  to  understand  the  impact  of  poor  data  quality  on 
the  enterprise.  The  second  is  to  develop  an  overall  strategy 
for  addressing  the  enterprise's  data  quality  problem.  The  third 
is  to  develop  and  deploy  a  data  quality  policy.  The  fourth  is 
to  lead  the  change.  Finally,  the  fifth  is  to  support  those  who 
will  lead  individual  projects. 

Deming  warned  that  the  failure  of  American  business  to 
realize  the  benefits  of  his  theory  were  due  to  leadership  paying 
lip  service  to  the  ideas  without  conviction  of  the  spirit.  This 
has  a  corollary  in  the  TDQM  environment.  Redman  (1986)  stated  a 
common  deadly  sin  was  for  leaders  to  proclaim  that  data  quality 
was  everyone's  responsibility.  This  proclamation  without  a 
clear  sense  of  direction  and  purpose  simply  lead  to  confusion 
and  inaction. 

Wang  and  Redman  also  indicate  that  there  is  another 
fundamental  change  that  must  take  place,  besides  just  realizing 
that  data  quality  is  important  to  the  enterprise.  The 
enterprise  must  change  its  view  of  data.  Data  must  be 
considered  inputs  to  a  product.  The  product  would  be  quality 
information.  Typically,  information  is  viewed  as  a  by-product 
of  congealed  data  derived  from  core  business  processes. 

To  put  action  to  words  and  to  begin  viewing  information  as 
a  product  we  can  use  Wang's  (1998)  TDQM  model  (See  Figure  3) . 
Wang  provides  operational  definitions  for  each  of  the  steps  in 
the  model.  The  first  step,  "Define"  consists  of  three  subtasks. 
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Figure  3.  Wang's  TDQM  Model. 


The  first  subtask  is  to  identify  elements  of  data  that  are 
needed  by  the  information  consumers.  This  is  usually  done 
through  the  aid  of  an  Entity-Relation  (ER)  diagram.  The  second 
subtask  is  to  determine  the  level  of  quality  needed  for  the 
information.  The  third  subtask  is  to  define  the  information 
processes.  Wang  labels  this  the  "Information  Manufacturing 
System."  Redman  (1996)  describes  it  as  the  "Information 


Chain" . 
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The  second  task  is  to  measure.  Measurement  comes  through  the 
development  of  information  quality  metrics.  Redman  utilizes  the 
traditional  Deming  approach  of  Statistical  Process  Control 
(SPC) .  Redman  applies  metrics  to  the  information  process  to 
establish  control  and  then  to  move  the  system  in  the  desired 
direction.  This  approach  has  a  heavier  focus  on  the  information 
systems  and  data  transfer  between  information  systems.  Wang 
(Haung  et .  al,  1999)  broadens  the  scope  of  the  metrics  and 
shifts  the  focus  to  the  end  users  or  consumers  of  the 
information  chain. 

Analysis,  the  third  task,  entails  discovering  the  root 
causes  for  the  poor  information  quality  and  calculating  the  cost 
of  the  poor  information  quality  to  the  organization.  The 
improvement  phase  may  utilize  a  variety  of  approaches.  One 
approach  is  to  modify/change  the  information  technology  (IT) . 
This  is  the  typical  approach  if  the  process  team  is  headed  by 
the  IT  department.  However,  the  IT  approach  should  be 
considered  in  the  context  of  the  larger  solution.  For  sustained 
improvement  to  occur  two  things  must  happen:  First,  the 
information  process  must  be  derived  from  a  business  requirement; 
secondly,  business  practices  must  be  aligned  to  the  information 
process  . 

Data  Models 

This  study  requires  a  further  exploration  of  the 
differences  between  data  quality  and  information  quality.  This 
exploration  will  take  place  through  the  description  of  data 
models  and  information  models.  Like  the  differences  between 
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data  quality  and  information  quality,  these  terms  are  not 
clearly  delineated  by  the  literature.  However,  observation  of 
the  MHS  leads  the  author  to  conclude  that  it  is  important  to 
make  a  distinction.  This  section  begins  with  a  very  simple 
diagram  of  a  data  model. 

Figure  4  is  a  simple  data  flow  model  depicting  a  real  world 
state  and  three  connected  information  system  (IS)  states.  In 
the  real  world,  there  are  thousands  of  entities  with  thousands 
of  characteristics  per  entity.  The  example  data  model  is  only 
concerned  with  the  "Patient"  entity  and  with  five 
characteristics  of  that  entity.  The  first  IS  data  model  (IS  1) 
depicts  how  it  will  store  the  real  world  information  in  its 
systems.  Age  is  recorded  in  a  format  familiar  to  all,  i.e. 
years.  The  format  for  storing  weight  in  IS  1  is  in  pounds,  but 
IS  2  and  IS  3  stores  the  information  in  kilograms.  The  gender 
of  the  patient  is  converted  to  a  numeric  binary  field  (0  = 
Female,  1  =  Male)  format  in  IS  1,  converted  to  a  text  field  (F  = 
Female,  M  =  Male)  in  IS  2,  and  converted  again  back  to  the 
binary  field  in  IS  3. 

The  simple  data  model  provides  an  opportunity  to  explore 
data  quality  problems.  The  most  obvious  error  is  that  IS  3  has 
changed  the  height  of  the  patient  from  68"  to  69".  This  is  a 
data  corruption  problem. 

The  not  so  obvious  error  is  in  the  smoking  status  of  the 
individual.  The  data  model  for  each  of  the  information  systems 
is  depicted  in  Table  3.  Remember  from  Figure  4,  that  the 
patient  is  a  smoker.  Figure  4  shows  that  all  three  information 
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Real  World 

IS  1 

IS  2 

IS  3 

1  Real  World 

IS  1 

IS  2 

IS  3  1 

Age:  35 

35 

35 

35 

Sex:  Male 

1 

M 

1 

Wt:  180  lbs 

180  lbs 

81.8 

81.8 

Ht:  5ft  8in 

68 

68 

69 

Smoker 

1 

1 

1 

Figure  4.  Simple  Data  Flow  Model 


Table  3 

Example  Smoking  Status  Data  Model 


Information  System 

IS  Rule  for  Capturing 

Smoking  Status 

Smoker 

Non-Smoker 

IS  1 

1 

0 

IS  2 

0 

1 

IS  3 

0 

1 

Note:  A  "0"  would  indicate  the  absence  of  the  attribute,  a  "1" 
would  indicate  the  presence  of  the  attribute. 
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systems  recorded  the  smoking  status  of  the  patient  as  "1."  If 
IS  2  or  IS  3  are  queried  for  the  smoking  status  of  the  patient, 
they  would  report  that  the  patient  was  a  non-smoker.  The 
problem  is  that  IS  2  failed  to  convert  the  data  to  comply  with  a 
change  in  data  model  rules.  This  example  points  out  two 
important  concepts.  To  improve  the  product  of  this  system  one 
must  know  the  data  model  for  each  IS  and  the  data  rules  for  how 
it  handles  each  characterist ic . 

Unfortunately,  few  things  in  the  real  world  are  simple 
models.  The  MHS,  as  one  of  the  largest  health  care 
organizations  in  the  world,  probably  has  one  of  the  largest 
medical  information  systems  in  the  world.  Figure  5  demonstrates 
the  complexity  of  the  system.  Figure  5  is  a  simplified  diagram 
of  one  of  the  five  major  IS'  in  the  MHS.  It  is  a  representation 
of  the  Ambulatory  Data  System  (ADS)  which  collects  outpatient 
encounter  information.  The  ADS  is  not  a  single  database.  At 
its  source,  the  data  model  is  designed  to  merge  data  from  other 
information  systems  (e.g.,  the  Composite  Health  Care  System 
(CHCS)  and  the  Defense  Enrollment  Eligibility  Reporting  System 
(DEERS) )  with  data  gleaned  from  a  single  patient  encounter 
occurring  in  one  of  the  thousands  of  MTF  clinics  worldwide. 

This  encounter  data  is  captured  on  an  optical  scanner  reader 
form,  or  entered  directly  into  an  MTF  level  ADS  database  through 
a  computer  interface.  A  subset  of  this  data  is  captured  in  a 
data  model  called  the  Standard  Ambulatory  Data  Record  (SADR) . 

The  SADR  is  either  pushed  or  pulled  (depending  on  location)  to 
an  IBM  feed  node,  archived,  and  then  sent  to  four  separate 


Figure  5.  ADS  Data  Flow  Diagram. 
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four  separate  information  processing  chains.  Each  IS  processing 
chain  then  takes  the  SADR  and  feeds  it  to  multiple  other 
computer  systems.  The  purpose  of  the  four  separate  processing 
chains  is  to  provide  a  data  model  appropriate  for  different  end 
users . 

Redman  (1986)  states  that  most  data  errors  occur  at 
information  system  interfaces.  Figure  5  demonstrates  the  almost 
unfathomable  number  of  interfaces  just  in  this  one  system.  The 
system  is  so  complex  that  the  problem  of  field  level  error  is 
almost  totally  ignored  by  the  TMA.  The  focus  of  TMA  data 
quality  efforts  is  on  ensuring  that  there  are  an  equal  number  of 
SADR  record  counts  between  each  system.  The  most  obvious 
problem  encountered  by  users  of  the  MHS  data  systems  is  that  the 
number  of  patients  seen  by  an  MTF  for  a  specified  time  frame 
varies  from  one  end  user  IS  to  the  next.  The  identified  culprit 
is  the  SADR.  They  are  either  dropped  or  duplicated  between 
interfaces . 

To  solve  this  "data  quality"  problem  the  MHS  engages  in 
what  Redman  terms  "database  bashing".  Database  bashing  is 
comparing  one  database  contents  against  another  to  identify  lost 
or  corrupted  field  level  data.  At  the  TMA  level  only  record 
count  integrity,  or  database  integrity,  is  considered.  The  TMA 
and  various  other  agencies  will  compare  one  end  user  database 
against  another  and  discover  that  the  SADR  record  counts  are 
different.  The  number  of  records  is  important,  because  it  is 
used  as  an  indicator  of  workload.  The  TMA  will  engage  in  a  time 
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and  manpower  intensive  effort  to  find  which  end  user  system  is 
the  most  incorrect.  No  one  is  completely  confident  of  any  system 
because  the  data  are  not  evaluated  with  the  real  world  model. 
Records  are  then  removed  or  repopulated  in  the  most  errant 
system  until  the  record  counts  between  the  two  systems  are 
equal . 

Redman  quoted  Mark  Twain  to  demonstrate  the  futility  of 
database  bashing,  "  A  man  with  one  watch  knows  what  time  it  is. 

A  man  with  two  watches  is  never  sure."  The  Air  Force  spent 
approximately  half  a  million  dollars  to  design  a  web  site  and 
institute  a  process  which  would  measure  and  improve  the  process 
of  SADR  record  transmittal  from  the  MTF  level  to  the  IBM  feed 
archive  node.  The  Air  Force  daily  engages  in  a  manpower  and  IT 
intensive  effort  just  to  ensure  counts  are  correct. 
Unfortunately,  this  process  does  not  ensure  that  the  counts  are 
maintained  all  the  way  to  the  end  user  systems. 

Record  count  integrity  does  not  ensure  field  level 
integrity.  The  author  does  not  find  any  comprehensive  study 
within  the  MHS,  which  has  addressed  the  issue  of  field  level 
integrity  in  the  SADR's.  Thus  even  if  the  MHS  is  able  to 
achieve  a  state  of  having  equal  record  level  counts  across  all 
systems  it  is  unlikely  that  the  fields  between  the  systems  would 
contain  the  same  data. 

To  summarize,  a  data  model  describes  the  storage  of 
characteristics  of  an  entity  in  an  information  system  and  the 
flow  of  these  characteristics  across  information  systems.  A 
data  model  allows  one  to  conceptually  view  data  quality  as  the 
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equality  of  data  between  information  system  states,  beginning 
with  the  real  world  state.  In  a  relatively  simple  data  model, 
measuring  and  improving  data  quality  is  a  time  and  labor 
intensive  task.  In  a  system  as  complex  as  the  MHS,  data  quality 
is  currently  confined  to  measuring  and  improving  database 
integrity . 

Information  Model 

An  information  model  is  more  complex  than  a  data  model. 

The  purpose  of  an  information  model  is  to  describe  the  processes 
involved  in  the  gathering  of  information.  The  information  model 
begins  with  the  end  user.  It  begins  with  the  question:  "What 
information  does  the  end  user  need  to  make  a  decision?"  and  then 
describes  all  of  the  processes  required  to  gather  and  display 
the  data  that  forms  the  required  information.  Figure  6  provides 
a  conceptual  view  of  a  generic  information  model. 

The  information  model  begins  by  identifying  the  data  the 
end  user  requires.  These  requirements  are  inclusive  of  all 
elements  needed  to  ensure  information  quality.  Using  Wang's 
definition  of  information  quality,  we  know  that  these  elements 
will  include  things  related  to  the  domains  of  intrinsic  quality, 
contextual  quality,  representational  quality,  and  accessibility 
quality . 

The  information  model  proceeds  to  define  the  processes, 
which  are  required  to  gather  the  data.  Internal  to  the 
information  model  is  the  data  model.  The  data  model  as 
previously  explained  will  describe  how  the  data  is  stored  in 
information  systems  and  how  it  flows  between  information 
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systems.  An  important  aspect  of  the  information  model  is  the 
recognition  that  there  are  processes  involved  in  moving  the 
information  between  systems.  The  next  node  in  the  model 
describes  all  of  the  processes  related  to  displaying  data  in  the 
form  of  information.  The  end  result  is  an  information  product. 
The  information  product  can  then  be  evaluated  by  the  same 
elements  that  helped  determine  the  requirements. 


Figure  6.  Information  Model 


Expanding  on  the  information  model,  there  are  two  different 
groups  involved  in  producing  an  information  product--the 
producers  and  the  custodians.  The  information  producers  are 
those  who  create  or  gather  data.  In  the  MHS,  the  information 
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producers  are  the  MTF  level  personnel.  The  custodians  are  those 
who  design,  develop,  and  maintain  the  information  system 
infrastructure  that  captures,  moves,  and  displays  the  data.  The 
custodians  of  MHS  data  are  spread  throughout  the  entire  system. 

Identifying  root  causes  for  the  deficient  information 
quality  within  MHS  may  be  accomplished  through  the  use  of  the 
information  model.  Defining  end  user  requirements  is  the 
starting  point.  Defining  requirements  is  never  an  easy  task. 
Often  the  end  users  are  only  vaguely  aware  of  what  they  really 
need.  It  has  been  said  that  the  MHS  data  systems  were  designed 
to  answer  any  question  that  the  GAO  or  other  government  official 
may  ask  the  MHS  leaders.  Like  any  system  produced  from  such 
broad  requirements  it  is  a  jack-of-all-trades  and  master-of- 
none  . 

The  second  root  cause  is  process  variation.  Deming 
postulated  that  the  Achilles  heel  of  quality  is  variation.  A 
convenient  example  is  the  ADS.  Please  refer  back  to  Figure  5. 

ADS  Data  Flow  Diagram.  The  process  begins  with  an  ADS  data 
entry  form  (either  electronic  or  hardcopy)  being  produced  from 
data  stored  in  DEERS  and  CHCS .  Thus,  two  potential  sources  of 
variation  and  error  are  immediately  identified.  The  MTF 
healthcare  provider  collects  the  patient  encounter  information 
on  a  bubble  sheet  or  through  a  computer  interface.  It  is  not 
unreasonable  to  hypothesize  that  each  provider  over  the  course 
of  a  day  worth  of  patients  will  have  some  variation  in  how 
he/she  enters  the  data.  A  further  progression  in  this 
hypothesis  is  that  a  group  of  providers  in  a  given  clinic  will 
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have  a  slightly  different  process  for  entering  the  data.  Each 
clinic  within  an  MTF  may  have  a  slightly  different  process  for 
using  ADS,  they  most  certainly  will  if  they  are  still  using  the 
manual  input  forms.  It  is  known  that  each  MTF  has  a  different 
process  for  sending  the  SADR  up  the  chain.  The  process  for 
moving  the  data  from  the  archive  node  to  each  of  the  four 
separate  end  user  systems  is  different.  The  process  for  moving 
the  data  from  each  node  within  the  end  user  systems  contains 
variation.  Thus  by  the  time  the  data  flows  from  collection  to 
output  the  amount  of  system  variation  it  is  subjected  to  is 
nearly  overwhelming. 

Try  to  visualize  the  variation  as  a  river.  The  variation 
begins  as  a  very  small  mountain  stream  with  just  a  trace  of 
sediment.  As  more  feeder  streams  are  added  to  the  system 
(providers,  MTF's,  transfer  points)  the  stream  becomes  wider, 
deeper,  and  muddier.  By  the  time  the  end  user  looks  at  the 
data,  he  is  confronted  with  a  raging  turbid  river  of  variation. 
If  this  river,  along  with  several  other  rivers  of  like  size  and 
turbidity,  empties  into  a  basin  (a  data  warehouse)  the  result  is 
not  a  pristine  mountain  lake;  it  is  a  stinking,  swirling 
cesspool  fraught  with  dangerous  undertow  currents. 

If  the  view  is  taken  that  information  is  a  product  derived 
from  a  system,  then  it  is  easier  to  comprehend  that  the  myopic 
focus  on  data  quality  is  insufficient  for  creating  information 
quality.  Data  quality  is  expressed  through  the  data  model. 
Information  quality  is  expressed  through  the  information  model, 
of  which  the  data  model  is  only  a  component.  The  TMA's  current 
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approach  to  data  quality  and  information  systems  architecture  is 
ignoring  end  user  requirements  and  system  variation. 

Accountability  is  a  major  element  missing  from  the  MHS 
information  model.  The  information  producers  at  the  MTF  level 
are  rarely  accountable  for  the  accuracy  or  timeliness  of  data 
they  feed  into  the  source  systems.  The  information  custodians 
are  rarely  accountable  for  the  design  of  systems,  which  assists 
the  producers  or  the  consumers.  Custodians  strongly  oppose 
taking  on  responsibility  for  the  quality  of  the  information  fed 
into  their  systems.  Alas,  it  appears  that  no  one  is  accountable 
for  controlling  even  the  most  obvious  variation  in  business 
procedures . 

Wang  and  Redman  throughout  their  publications  point  to  the 
requirement  for  the  information  process  to  be  managed  in  its 
entirety.  Depending  on  the  academic  school,  the  person  or  group 
upon  whom  this  responsibility  falls  is  labeled  the  process 
manager (s)  or  process  owner (s).  Whichever  label  is  chosen,  the 
key  concept  is  accountability.  An  information  process  manager 
is  a  product  manager.  These  individuals  manage  the  information 
model  like  any  other  production  model  and  consequently  are 
accountable  for  the  quality  of  the  information  product. 

To  summarize,  an  information  model  provides  a  systems  view 
for  information  quality.  The  model  treats  information  as  a 
product  derived  from  well  defined  processes,  managed  by  an 
accountable  individual  or  group.  The  information  model 
describes  a  production  process.  Information  systems  and 
technology  are  important  components  of  the  model,  but  not  the 
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models  primary  focus.  The  author  finds  the  information  model 
vastly  superior  to  the  data  model  in  describing  the  needs  of 
consumers  and  producers . 


PURPOSE 

The  MHS  is  tasked  with  providing  the  22  HEDIS  Medicare 
measures  listed  in  Table  4  to  HCFA  (HCFA,  1999) .  Unfortunately, 
it  is  not  possible  within  the  parameters  of  this  study  to  assess 
the  entire  list  of  measures.  This  study  will  limit  its  scope  to 
just  one  measure:  Beta  Blocker  Treatment  After  a  Heart  Attack. 

For  the  targeted  measure,  this  study  will  define  the  current 
information  model  for  collecting  data  and  reporting  the 
measurement  to  HCFA.  It  is  known  from  previous  studies  (VRI, 
1998b)  that  the  collection  of  this  measure  has  been  difficult 
for  the  MHS.  Therefore,  this  study  will  formulate  an 
alternative  information  model.  Finally,  this  study  will  compare 
the  two  models  using  Wang's  dimensions  of  information  quality. 

It  is  hypothesized  that  a  new  information  model  that  treats 
the  TSP  HEDIS  Medicare  measure  as  an  information  product  will  be 
vastly  superior  to  the  current  information  model  in  the  MHS. 

The  alternative  model  developed  in  this  study  is  hoped  to 
provide  MHS  leaders  with  a  guide  to  building  a  capable 
information  system.  The  results  may  also  be  used  by  MHS  leaders 
as  a  template  in  developing  a  robust  system  wide  information 
quality  program  to  ensure  the  survivability  of  one  of  the  worlds 
largest  managed  care  organizations. 
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Table  4 

HEDIS  2000  Medicare  Measures 


Domain 


Measure 


Effectiveness  of  Care: 

•  Antidepressant  Medication  Management  (for  those  with  a  drug 
benefit) 

•  Cholesterol  Management  After  Acute  Cardiovascular  Events 

•  Breast  Cancer  Screening 

•  Beta  Blocker  Treatment  After  A  Heart  Attack 

•  Comprehensive  Diabetes  Care 

•  Follow-up  After  Hospitalization  for  Mental  Illness 

•  Controlling  High  Blood  Pressure 

•  Medicare  Health  Outcomes  Survey 
Access  to/ Availability  of  Care: 

•  Adults'  Access  to  Preventive/Ambulatory  Health  Services 

•  Availability  of  Language  Interpretation  Services 

Use  of  Services: 

•  Frequency  of  Selected  Procedures 

•  Inpatient  Utilization  -  General  Hospital/ Acute  Care 

•  Ambulatory  Care 

•  Inpatient  Utilization  -  Non- Acute  Care 

•  Mental  Health  Utilization  -  Inpatient  Discharges  and  Average  Length 
of  Stay 

•  Mental  Health  Utilization  -  Percentage  of  Members  Receiving 
Inpatient,  Day /Night  and  Ambulatory  Services 

•  Chemical  Dependency  Utilization  -  Inpatient  Discharges  and  Average 
Length  of  Stay 

•  Chemical  Dependency  Utilization  -  Percentage  of  Members 
Receiving  Inpatient,  Day/Night  and  Ambulatory  Services 

•  Outpatient  Drug  Utilization  (for  those  with  a  drug  benefit) 

Health  Plan  Descriptive  Information: 

•  Board  Certification/Residency  Completion 

•  Total  Enrollment  by  Percentage 

•  Enrollment  by  Product  Line  (Member  Years/Months ) 

Note:  Adapted  from  Health  Care  Financing  Agency  (HCFA)  (1999) . 

Operational  Policy  Letter  #110  (OPL99.110) .  Baltimore,  MD : 


Author . 
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CHAPTER  2 

METHODS  AND  PROCEDURES 

Methodology 

This  study  will  use  a  case  study  methodology.  In  describing 
the  specifics  of  this  approach,  the  first  step  will  be  to 
describe  the  current  information  model  for  the  collection  of  the 
data  required  by  the  targeted  measure.  This  step  will  include 
the  identification  and  description  of  the  following  information 
model  elements:  the  consumers,  the  producers,  the  custodians, 
the  information  systems,  the  information  manager (s)  and  the 
current  process,  beginning  with  the  initial  collection  of  the 
data  through  to  the  reporting  the  completed  measure  to  HCFA. 

The  second  step  will  be  to  evaluate  this  model  using  Wang' s 
attributes  of  information  quality:  Intrinsic  IQ,  Contextual  IQ, 
Representational  IQ,  and  Accessibility  IQ.  The  final  step  will 
be  to  formulate  an  alternative  model  that  provides  for  a  higher 
quality  information  product. 

HEP I S  2000  Technical  Requirements 

HEDIS  measurement  requires  the  application  of  very  complex 
formulae  to  MCO  datasets.  To  become  acquainted  with  the 
complexities  involved  in  HEDIS  measurement,  this  section  will 
define  the  technical  requirements  for  the  targeted  measure.  In 
general,  the  measure  is  designed  to  evaluate  appropriate  follow¬ 
up  care.  The  technical  requirements  form  the  cornerstone  of  the 
end  user  requirements  in  the  information  models.  The  HEDIS  2000 
Technical  Specifications  (NCQA,  1999)  specify  in  precise  detail 
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the  denominator  and  numerator  for  the  measure:  Beta  Blocker 
Treatment  After  a  Heart  Attack. 

Denominator 

The  criteria  for  inclusion  in  the  denominator  (target 
population)  are  as  follows:  1)  all  members  35  years  and  older  as 
of  December  31  during  the  measurement  year;  2)  members  who  were 
hospitalized  and  discharged  alive  from  January  1  through 
December  24  of  the  measurement  year  with  a  diagnosis  of  acute 
myocardial  infarction  (AMI)  (ICD-9CM  code  410. xl  with  "x"  equal 
to  any  digit) ;  3)  members  who  received  an  ambulatory 
prescription  for  beta  blockers  upon  discharge.  The  following 
clauses  also  apply:  1)  the  member  must  have  no  gaps  in 
enrollment  during  the  measurement  year  and  must  have  stayed 
enrolled  in  the  plan  at  least  seven  days  after  discharge;  2)  if 
a  member  had  more  than  one  episode  of  AMI  during  the  measurement 
year,  then  only  the  first  episode  is  counted;  3)  members'' 
transferred  to  acute  care  facilities  after  December  24th  are  to 
be  excluded;  and,  4)  if  the  member  was  re-admitted  to  an  acute 
or  non-acute  care  facility  for  any  diagnosis  within  seven  days 
after  discharge,  the  member  is  excluded  from  the  denominator. 

Additionally,  the  NCQA  strongly  recommends  that  the  MCO's 
exclude  from  the  denominator  those  members  who  are  identified  as 
having  had  a  contradict ion  to  beta  blocker  therapy.  Per  the 
NCQA  (1999),  the  following  conditions  allow  exclusion:  insulin 
dependent  diabetes  mellitus,  heart  block  >1  degree,  sinus 
bradycardia,  heart  failure,  left  ventricular  dysfunction,  and 
chronic  obstructive  pulmonary  disease  (COPD) .  Also,  use  of  the 
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following  prescriptions  by  the  member  indicate  a 
contraindication  to  beta  blocker  therapy:  insulin,  inhaled 
cort icosteriods ,  or  leukotrien  antagonists. 

Numerator 

The  numerator  for  the  measure  is  derived  from  members  who 
receive  an  ambulatory  prescription  for  beta  blockers  within 
seven  days  after  discharge.  A  general  list  of  allowable  beta 
blockers  is  in  Table  5;  a  complete  list  of  allowable  medications 
may  be  found  on  the  NCQA  website  http://www.ncqa.org.  The 
prescription  may  be  filled  on  an  ambulatory  basis  anytime  during 
hospitalization,  up  to  seven  days  after  discharge.  If  the  MCO 
is  unable  to  determine  if  the  prescription  is  filled  on  an 
ambulatory  basis  during  the  admission,  only  those  members  with 
prescriptions  filled  within  seven  days  of  discharge  are 

Table  5 

NCQA  Allowable  Beta  Blockers 


Acebutolol  HCL 
Atenolol 
Betaxolol  HCL 
Bisprolol  Fumarate 
Carteolol  HCL 


Carvedilol 
Labetalol  HCL 
Metoprolol  Succinate 
Metroprolol  Tartate 
Nadolol 


Penbutolol  Sulfate 
Pindolol 
Propranolol  HCL 
Sotalol  HCL 
Timolol  Maleate 


Note:  Adapted  from  HEDIS  2000  Technical  Specifications,  Volume 


2,  by  the  NCQA,  1999.  Washington,  D.C:  Author. 
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allowable.  If  the  member  was  on  an  active  beta  blocker 
prescription  at  the  time  of  admission,  then  the  member  is  to  be 
included  in  the  numerator.  Members  transferred  to  acute  care 
facilities  require  additional  scrutiny.  Inclusion  in  the 
numerator  requires  the  MCO  to  determine  if  the  member  was  on  a 
prescription  from  30  days  prior  to  admission  through  7  days 
after  discharge. 

Primary  data  collection  was  beyond  the  scope  and  resources 
of  this  study.  The  study  design  utilized  modeling  and  analysis 
to  derive  its  results.  Technical  specifications  were  provided 
for  the  purpose  of  demonstrating  the  end  user  requirements  and 
also  to  give  a  glimpse  into  the  robust  data  model  requirements 
for  the  information  system. 
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CHAPTER  3 

CURRENT  INFORMATION  MODEL 


Figure  7.  HEDIS  Data  Flow 
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Current  Process  for  Collecting  Data 

Establishing  a  current  data  flow  diagram  for  any  MHS  data 
system  is  an  arduous  task.  Considering  the  turmoil  within  the 
MHS  information  management  program  offices  over  the  last  six 
months,  what  is  current  will  depend  on  the  source  and  can 
literally  change  within  hours.  Nevertheless,  using  data  experts 
within  VRI  and  TMA  documentation  (TMA,  2000),  Figure  7  was 
developed  to  provide  a  macro  view  of  the  data  flows  related  to 
the  targeted  measure.  The  three  major  processes  that  must  be 
mapped  are  enrollment,  claims/clinical  data  flow,  and  HEDIS 
report  generation. 

The  first  major  step  in  the  data  flow  process  will  begin 
with  an  eligible  beneficiary  enrolling  into  the  TSP,  a  data  flow 
diagram  for  this  is  found  in  Appendix  1 .  The  process  begins 
with  a  beneficiary  sending  a  request  for  enrollment  into  TSP  to 
a  Managed  Care  Support  Contractor  (MCSC) .  The  MCSC  forwards  the 
request  to  the  Medicare  Processing  Center  (MPC) .  The  MPC  was 
developed  and  is  run  by  Litton  PRC,  a  civilian  contractor  to  the 
MHS.  The  MPC  does  an  eligibility  check  on  the  applicant 
utilizing  the  Defense  Eligibility  Enrollment  Reporting  System 
(DEERS) .  The  MPC  then  forwards  eligible  enrollee  applications 
to  HCF A .  HCFA  reports  back  to  the  MPC  on  Medicare  eligibility. 
The  MPC  forwards  the  Medicare  eligibility  status  back  to  the 
MCSC,  which  enters  it  into  the  Composite  Health  Care  System 
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(CHCS) .  CHCS  is  then  used  to  update  the  DEERS  file  with  a 
special  TSP  Medicare  eligible  identifier.  The  CHCS  also 
forwards  the  information  to  CEIS.  This  process  is  critical  for 
HEDIS  Medicare  reporting  because  it  provides  the  only  unique 
identifier  for  the  TSP  population  within  the  MHS .  It  is  also 
critical  because  HEDIS  reporting  requires  patient-level  data  be 
identified  by  the  HCFA  provided  health  insurance  claim  (HIC) 
number . 

The  second  major  step  in  the  overall  process  is  to  identify 
how  the  claims  and  clinical  data  flow  through  the  systems. 

Figure  7  begins  with  an  enrollee  being  admitted  to  a  facility 
with  the  diagnosis  of  AMI.  The  enrollee  may  be  admitted  either 
to  an  MTF ,  a  network  facility,  or  a  non-network  facility.  The 
data  flows  are  vastly  different  depending  on  location.  A 
diagram,  which  focuses  exclusively  on  this  process,  may  be  found 
in  Appendix  2 . 

The  MTF  relies  upon  CHCS  to  produce  a  Standard  Inpatient 
Data  Record  (SIDR) ,  drawing  upon  data  elements  contained  within 
DEERS.  For  the  purposes  of  the  target  measure  the  SIDR  will 
only  contain  information  related  to  patient  identifier,  date, 
and  diagnosis.  CHCS  is  used  to  send  the  required  pharmaceutical 
data  up  the  information  chain  in  another  format  separate  from 
the  SIDR,  referred  to  as  HL7  (Health  Level  7)  feeds.  These 
feeds  are  collected  at  the  TRICARE  Management  Activity  West,  in 
Aurora,  Colorado  (TMA  West)  in  an  information  system  called  the 
IBM  Mainframe.  The  TMA  Office  of  Acquisition  Management  and 
Support  (TMA  AM&S)  controls  this  system.  CEIS  accesses  this 
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data  via  the  Integrated  Database  Replacement  (IDB-R)  and  is 
required  to  convert  the  SIDR  and  HL7  data  to  civilian  industry 
standardized  codes  (UB92 ' s  and  HCFA  1500's)  and  forward  this 
data  to  the  MPC. 

If  the  enrollee  was  admitted  at  a  network  or  non-network 
facility,  then  the  MCSC  forwards  the  data  by  two  separate 
processes  to  two  distinct  locations.  One  data  feed  in  an  MHS 
format  is  sent  via  the  form  of  a  Health  Care  Standardized  Record 
(HCSR)  which  goes  to  the  IBM  Mainframe  at  TMA  West.  The  HCSR 
data  is  used  to  populate  the  CEIS,  thus  allowing  CEIS  users  to 
view  non-MTF  provided  services.  The  second  feed  consisting  of 
industry  standard  codes  in  the  form  of  UB92s  and  HCFA  1500's  is 
sent  directly  to  the  MPC.  The  MPC  is  tasked  with  submitting  the 
converted  HCSR  data  (MTF  care) ,  the  network  claims,  and  non¬ 
network  claims  to  HCFA  in  a  civilian  industry  format. 

The  third  major  step  is  to  identify  how  the  HEDIS  Medicare 
reports  are  produced.  VRI  is  the  contractor  of  choice  for 
producing  the  HEDIS  reports.  VRI  is  granted  access  under 
contract  to  query  the  SAS  datasets  located  on  the  Ft.  Detrick 
mainframe.  Ft.  Detrick  receives  data  feeds  from  the  Residual 
Legacy  Processing  (RLP)  at  TMA  West.  These  feeds  contain  SIDR 
data  collected  from  the  MTF ' s  by  CEIS,  HCSR  data  from  the 
Information  Management  Technology  and  Reengineering-Aurora 
(IMT&R-A)  and  processed  by  CEIS,  and  population  data  from  a 
DEERS  population  extract  (VRI,  1999) . 

Noticeably  absent  from  this  feed  list  is  HL-7  or  ancillary 
care  data,  which  includes  pharmacy  data.  Currently  a  definitive 
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corporate  repository  of  ancillary  data  is  not  available.  The 
ancillary  data  retained  within  the  IDB-R,  after  a  monthly  update 
to  the  datamarts,  is  in  an  archived  format  that  greatly  hinders 
abstraction  and  access. 

VRI  Statistical  Analysis  Software  (SAS)  programs  access  the 
datasets  at  Ft.  Detrick  and  through  complex  coding  algorithms 
extract  the  required  information  into  flat  files,  which  are 
uploaded  to  a  VRI  Access®  database.  The  VRI  Access®  database  is 
then  queried  to  produce  the  required  data  for  HEDIS  report 
generation . 

Obviously  without  pharmacy  data,  the  target  measure  is  not 
reportable  using  only  administrative  sources.  (HCFA  and  the 
NCQA  provide  for  a  modified  method  of  collecting  the  measure 
through  sampling  and  medical  record  abstraction.  However,  this 
method  is  beyond  the  scope  of  this  study  and  is  not  related  to 
the  study  question.)  There  are  other  potential  sources  of  data. 
The  MPC  Beneficiary  File  maintained  by  LITTON  PRC,  may  contain 
many  of  the  required  elements.  Additionally,  there  is  the  All 
Regions  Server  (ARS)  which  is  maintained  by  SAIC,  another  MHS 
civilian  contractor.  The  ARS  is  reported  to  contain  detailed 
National  Mail  Order  Pharmacy  (NMOP)  data  at  the  beneficiary 
level  and  summary  level  (non-beneficiary  level)  information  from 
CEIS  for  data  validation  purposes.  Due  to  constraints  on 
access,  time,  and  funding  these  alternative  data  sources  have 
not  been  thoroughly  evaluated  by  the  MHS  for  their  potential  use 
in  HEDIS  reporting  and  this  evaluation  is  well  beyond  the  scope 
of  this  study. 
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With  the  information  flow  identified,  it  is  now  possible  to 
identify  the  consumers,  producers,  custodians,  information 
systems,  and  information  managers.  This  information  is  provided 
in  Table  6.  The  information  product  in  this  study  is  the  HEDIS 
Medicare  measure.  Obviously,  in  a  system  as  large  and  complex 
as  this,  there  are  many  intermediary  consumers  of  information. 
However,  only  the  TMA,  the  Lead  Agents,  HCFA,  and  the  NCQA 
Table  6 

Current  Information  Model  Entities 


_ Role _ 

Consumers : 
Producers : 
Custodians : 


Information 
Systems : 


Information 
Manager : 


Entity _ 

HCFA,  NCQA,  TMA,  Lead  Agents 

MTF  staff,  MCSC  staff,  MPC  staff,  VRI  staff 
U.S.  Army  Medical  Information  Systems  and 
Services  Agency  (Ft. Detrick  Mainframe) 

CHCS  Program  Management  Office 

TMA  AM&S  (IBM  Mainframe,  IDB-R,  RPU,  RLP ) 

CEIS  Program  Management  Office 
Defense  Manpower  Data  Center  (DEERS) 

Litton  PRC  (MPC) 

VRI  (HEDIS  Access  Database) 

DEERS,  CHCS,  MPC,  MCSC  Systems,  IBM  Feed  Node, 
IDB-R,  RPU,  RLP,  Ft.  Detrick  Mainframe,  CEIS, 
VRI  SAS  extracts,  VRI  Access  Database 

TMA  Office  of  Medical  Affairs  (TMA  OMA) ? ! 
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require  a  finished  product.  Like  consumers,  it  is  obvious  in 
this  system  that  there  are  a  number  of  producers  and 
intermediary  producers .  Those  of  greatest  concern  are  the 
producers  having  the  greatest  impact  on  the  finished  product. 
Producers  inputing  data  directly  into  an  information  system,  or 
those,  which  vastly  alter  received  data  to  supply  another 
output,  are  of  the  most  interest.  The  list  of  custodians  is 
very  impressive.  They  represent  politically  powerful  and  at 
times  conflicting  interests.  The  list  of  information  systems  is 
also  very  impressive.  Almost  every  transfer  point  will  involve 
the  conversion  or  manipulation  of  data  to  meet  the  data  model  of 
the  receiving  system.  The  final  task  was  to  identify  the 
Information  Manager.  The  TMA  Office  of  Medical  Affairs  was 
selected  as  the  Information  Manager,  only  because  they  are  the 
contracting  agency  for  the  HEDIS  report  generation. 
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CONCEPTUAL  HEDIS  DATA  FLOW 
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Conceptual  Model 

The  current  information  model  is  not  sufficient  to  report 
the  targeted  measure.  Therefore,  this  study  will  propose  an 
alternative  conceptual  model,  shown  in  Figure  8.  This  model 
utilizes  a  functional  versus  a  technical  approach.  The  chosen 
functional  approach  maps  the  requirements,  whereas  a  technical 
approach  maps  the  actual  data  elements  in  an  entity  relationship 
and  produces  the  data  model  for  each  information  system. 
Reguirement s 

Referring  back  to  Figure  6,  the  first  step  in  producing  the 
information  model  is  the  identification  of  end  user 
requirements.  The  requirements  are  found  in  detail  in  the  HEDIS 
2000  Technical  Specifications  (NCQA,  1999) .  Additionally,  since 
the  goal  is  to  increase  the  quality  of  the  product  it  will  be 
necessary  to  state  requirements  related  to  the  intrinsic, 
contextual,  representational,  and  accessibility  attributes  of 
the  information  model. 

To  increase  the  intrinsic  information  quality  of  the  model 
it  will  be  necessary  to  identify  methods  that  will  enhance  the 
accuracy  and  objectivity.  Unfortunately,  in  the  near  term  there 
is  no  strategy  to  combat  the  MHS  information  system  stigma. 
However,  building  quality  control  measures  into  the  system  may 
increase  the  accuracy  of  the  model.  Accuracy  and  accountability 
go  hand  in  hand.  Forcing  accountability  for  the  information 
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entered  into  the  system  down  to  the  MTF  level  will  greatly 
increase  the  accuracy  of  the  information.  To  address  the  issue 
of  objectiveness  it  will  be  necessary  for  the  MHS  to  get  over 
its  "we  are  unique"  syndrome.  If  the  MHS  requires  MCSC  to 
capture  the  information  in  UB92  and  HCFA  1500  formats,  it  would 
appear  prudent  for  the  MHS  to  do  the  same  for  care  provided 
inside  the  MTF. 

The  greatest  challenge  will  come  from  the  contextual  IQ 
domain.  There  are  currently  three  separate  processes  for 
receiving  ambulatory  prescriptions:  those  provided  at  the  MTF, 
those  provided  by  a  MCSC  or  network  pharmacy,  and  those  provided 
through  the  National  Mail  Order  Pharmacy  (NMOP)  program.  The 
challenge  is  getting  all  three  to  submit  their  data  to  the  same 
location  in  the  same  format. 

Representational  IQ  is  really  tied  into  the  above  intrinsic 
and  contextual  requirements.  The  representational  IQ  will  be 
greatly  increased  if:  1)  the  number  of  transfer  points  is 
decreased;  2)  the  number  of  separate  information  systems  is 
decreased;  3)  the  number  of  custodians  is  decreased;  and  4)  the 
information  captured  at  the  time  of  the  encounter  is  the  same 
and  in  the  same  formats  as  that  which  is  used  to  calculate  the 
HEDIS  measures. 

The  final  requirement  is  the  identification  of  an  office  at 
the  TMA  level  that  has  the  authority  and  accountability  to 
function  as  the  corporate  information  manager  for  the  TSP 
demonstration.  This  office  must  be  empowered  to  fund  the 
system,  control  access  and  content,  and  must  be  politically 
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powerful  enough  to  battle  the  MHS  bureaucracy.  This  office 
would  not  function  independently  but  instead  would  rely  upon 
operational  control  of  personnel  at  the  MTF  and  MCSC  sites  that 
function  to  support  the  mission  of  the  office. 

Process  for  Collecting 

The  process  for  collecting  the  data  must  be  well  defined 
and  standardized  throughout  the  healthcare  system.  The  process 
must  treat  the  data  as  a  product  and  not  as  a  by-product  of 
another  activity.  The  proposed  model  would  have  the  MTF/TMA 
personnel  enter  the  data  directly  into  the  MPC  system.  Thus, 
mirroring  the  processes  currently  in  place  at  the  MCSC' s .  The 
MTF' s  would  utilize  the  same  business  processes  as  the  MCSCs . 

The  MTF  would  consider  each  encounter  with  a  TSP  member  as  a 
business  transaction  that  requires  standardized  civilian 
documentation.  The  NMOP  would  also  be  required  to  enter 
prescription  data  in  the  same  format  and  via  the  same  systems  as 
used  by  the  MTF ' s  and  MCSC's.  To  build  quality  into  the 
process,  it  would  be  advantageous  to  have  TMA  TSP  Information 
representatives  at  each  demonstration  site,  each  MCSC,  the  MPC, 
and  working  with  the  NMOP. 

Data  Model 

The  data  model  is  beyond  the  scope  of  this  study.  It  would 
require  technical  analysis  of  the  MPC  and  systems  used  by  the 
MCSC.  The  ideal  data  model  would  greatly  reduce  the  number  of 
information  systems  involved.  Additionally,  it  would  contain  a 
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standard  civilian  based  data  model  that  is  directly  compatible 
with  HCF A  systems,  eliminating  the  need  for  conversion. 

Process  for  Retrieving 

Since  the  MPC  would  ultimately  contain  all  the  data  related 
to  the  members  in  the  TSP  demonstration,  it  follows  that  the 
HEDIS  reports  should  rely  upon  this  as  the  sole  source  for  data. 
The  contractor,  which  is  responsible  for  collecting  and 
reporting  the  measures,  would  develop  the  software  programs  that 
would  be  incorporated  into  the  MPC.  With  the  programs  imbedded 
into  the  MPC  it  would  eliminate  multiple  data  conversion  steps, 
representational  problems,  and  access  barriers.  Additionally, 
the  TMA,  Lead  Agents,  and  MTF ' s  could  track  the  status  of  the 
metric  throughout  the  year.  Thereby,  allowing  the  MHS  to 
possibly  institute  a  program  of  continuous  improvement  and  not 
be  surprised  by  a  yearly  report. 

The  MPC  system  would  also  be  the  functional  system  for 
tracking  enrollment  and  eligibility  status  by  the  MTF ' s  and 
MCSC's.  The  MCSC  will  need  to  initially  utilize  the  DEERS 
system  to  verify  eligibility.  However,  once  eligibility  is 
determined  the  only  system  that  would  need  to  be  updated  is  the 
MPC. 

Entity  Identification 

Table  7  provides  a  conceptual  view  the  systems  and  entities 
involved  in  the  new  information  flow.  The  numbers  of  data 
custodians  were  reduced  to  only  one,  namely,  the  MPC. 

The  only  information  system  to  be  used,  with  the  exception  of 
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the  initial  eligibility  check,  is  the  Medicare  Processing  Center 
system.  Lastly,  an  information  management  team  is  identified. 
They  will  be  held  directly  accountable  for  the  quality  of  the 
information  entered  into  and  maintained  by  the  system. 

Table  7 

Conceptual  Information  Model  Entities 

Role  Entity 

Consumers:  HCFA,  NCQA,  TMA,  Lead  Agents 

Producers:  MTF  staff,  MCSC  staff,  MPC  staff,  VRI  staff 

Custodians:  Medicare  Processing  Center 

Information 

Systems:  MPC,  (DEERS  for  initial  verification) 

Information 

Manager:  TMA  Corporate  TSP  Information  Center,  with 

offices  at  each  demonstration  MTF  site,  MCSC,  and 


MPC. 
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CHAPTER  5 
DISCUSSION 

This  study  developed  a  framework  for  evaluating  information 
models  through  an  extensive  literature  review.  The  evaluation 
framework  is  based  upon  the  Information  Quality  attributes 
identified  by  Wang  (Huang,  et .  al,  1999;  Wang,  Lee,  Pipino,  & 
Strong,  1998) .  The  four  attributes  were  assigned  a  subjective 
score  based  on  issues  identified  with  the  information  flow. 
Intrinsic  IQ  addresses  the  issues  of  accuracy,  objectivity, 
believability,  and  reputation.  Contextual  IQ  addresses  the 
issues  of  relevancy,  added  value,  timeliness,  and  completeness. 
Representational  IQ  addresses  the  issues  of  interpretability , 
ease  of  understanding,  conciseness  of  representation,  and 
consistency  of  representation. 

Since  it  was  discovered  that  the  current  information  model 
was  incapable  of  reporting  the  targeted  measure,  it  was 
necessary  to  produce  an  alternative  conceptual  information 
model.  The  study  will  now  turn  its  attention  to  utilizing  the 
framework  to  evaluate  both  the  current  information  model  and  the 
conceptual  information  model. 

Current  Model  Evaluation 

The  current  information  model  is  evaluated  utilizing  Dr. 
Wang's  attributes  of  information  quality.  The  results  are 
displayed  in  Table  8. 
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Table  8 

Current  Model  Evaluation 


IQ  Attribute  Score  Issues 


Intrinsic  IQ 


Contextual  IQ 


□  Historically,  these  data  have  a 
very  poor  reputation  among  GAO  and 
IG  auditors. 

□  Direct  MTF  patient  care  data 
require  a  conversion  process  from 
HCSR  format  to  UB92  and  HCFA  1500 
formats . 

□  Different  processing  data  flows 
for  MTF  and  non-MTF  provided  care. 

□  Lack  of  quality  improvement 
processes . 

□  No  identified  TSP  information 
process  owners  at  the  MTF  level. 

□  TMA  OMA  responsible  for  HEDIS 
reporting,  but  lack  authority  to 
control  information  process. 

•  Target  measure  is  not  producible 
using  only  administrative  data 
sources . 

•  Process  is  time  consuming,  measure 
can  only  be  produced  yearly. 

•  Data  are  incomplete,  markedly 
lacking  pharmacy  data. 


Representational  IQ  LOW  *  Data  have  muitiple  formats  in 

multiple  systems. 


Accessibility  IQ  LOW  ^  No  corporate  source  for  historical 

ancillary  data. 

^  Archived  historical  data  are 
extremely  difficult  to  access. 

^  Multiple  custodians  produce 

barriers  to  accessing  the  data. 
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Evaluating  the  current  information  model  utilizing  Wang's 
attributes  of  information  quality  finds  that  the  model  scores 
low  in  every  area.  Most  importantly,  it  fails  to  meet  the 
requirements  of  the  TSP  demonstration.  However,  even  if  it 
could  produce  a  product,  the  validity  of  the  product  would  be  in 
question  because  of  the  high  degree  of  variability  inherent  in 
the  model. 


Conceptual  Model  Evaluation 

It  is  impossible  to  actually  evaluate  the  product  of  this 
model,  however  an  analysis  of  the  conceptual  design  may  be 
conducted.  Using  Wang's  attributes  of  Information  Quality,  the 
results  of  the  conceptual  model  evaluation  are  found  in  Table  9. 

The  model  receives  high  scores  in  all  areas  except  the 
intrinsic  IQ  domain.  In  the  short  term,  it  is  impossible  to 
overcome  the  stigma  attached  to  MHS  systems  by  outside  entities. 
To  overcome  this  problem  the  accuracy  and  usefulness  of  the 
system  would  require  independent  evaluation. 

Information  Manager  Analysis 

In  addition  to  the  model  evaluations  already  provided,  there 
are  three  questions  that  MHS  information  managers  should  be 
asking . 
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Table  9 

Conceptual  Model  Evaluation 


IQ  Attribute  Score  Issues 


Intrinsic  IQ  MED  □  Historically,  MHS  data  systems 

have  a  very  poor  reputation  among 
GAO  and  IG  auditors. 

□  Direct  MTF  care  and  non-direct  MTF 
care  would  be  captured  in  the  same 
format . 

□  MTF  and  non-MTF  provided  care  have 
the  same  information  flow  process. 

□  Quality  improvement  processes  are 
built  into  the  system  by  providing 
direct  feedback  from  the  MFC. 

□  The  information  model  has  well 
identified  process  owners  at  all 
levels  of  the  system. 

□  TMA  would  have  operational  control 
over  the  process. 

Target  measure  would  be  producible 
using  only  administrative  data 
sources . 

Process  would  be  timely  and 
measure  could  be  reported  monthly. 
Data  would  be  complete. 

Representational  IQ  HIGH  ♦♦♦  pa-(:a  would  have  a  single  format  on 

a  single  system. 

Accessibility  IQ  HIGH  ^  There  would  be  a  corporate 

resource  for  all  data  related  to 
the  TSP  demonstration. 

^  Single  custodian  controls  access 
to  the  data. 


Contextual  IQ  HIGH  # 


Information  Quality  Model  67 


Does  the  model  treat  data  as  a  product  or  as  a  by-product  of  the 
organization  ? 

The  current  model  clearly  treats  data  as  a  by-product  of  its 
existing  operations.  It  is  apparent  from  the  current  model's 
design  that  the  MHS  expected  to  play  by  different  rules  than 
those  required  of  its  contractors.  The  MCSC's  must  treat  the 
data  as  a  product,  because  it  is  the  source  of  their  funding. 

In  the  business  world,  fiscal  remuneration  drives  the  design  of 
information  systems.  Conversely,  the  MHS  appears  to  have  hoped 
that  it  could  carry  out  a  civilian  business  model,  which 
Medicare  reimbursement  most  certainly  is,  without  instituting 
comparable  civilian  practices  into  its  internal  healthcare 
system  and  supporting  information  systems.  The  conceptual  model 
overcomes  this  bias  by  forcing  the  MHS  to  carryout  the  same 
business  processes  as  the  MCSC. 

Is  guality  built  into  the  system? 

After  multiple  attempts  to  get  a  handle  on  their  data 
quality  improvement,  the  MHS  remains  without  a  definitive, 
workable  program  in  place.  Corey  (1997)  described  three 
attributes  of  the  MHS  that  adversely  impacts  data  quality  and 
the  attempts  to  improve  the  data  quality:  organization,  process, 
and  personnel.  There  is  no  one  political  entity  that  has  the 
power,  and  or,  willingness  to  take  control  of  the  system.  The 
Services  refuse  to  accept  that  their  individual  desires  should 
be  subservient  to  an  MHS-wide  task.  Each  Service  wants  to 
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remain  in  control  of  their  individual  share  of  the  pie.  The 
conceptual  model  overcomes  this  inherent  nature  by  requiring 
that  a  TMA  level  office  have  direct  operational  control  over  the 
personnel  and  systems  handling  the  TSP  demonstration  data. 

Does  the  model  support  the  TSP  demonstration? 

The  current  model  does  not  fully  support  the  demonstration.  In 
fact,  the  current  model  may  well  be  a  major  reason  for  its 
downfall  judging  from  the  numerous  GAO  reports.  The  current 
model  is  unable  to  report  the  targeted  measure  in  this  study. 
Further,  based  on  past  studies  it  is  highly  unlikely  that  the 
current  model  will  succeed  in  supporting  many  of  the  HEDIS 
Medicare  measures.  The  conceptual  model  would  be  able  to  report 
the  targeted  measure  and  thus,  would  support  the  TSP 
demonstration . 
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Chapter  6 

CONCLUSSIONS  and  RECOMMENDATIONS 

This  study  has  evaluated  the  current  processes  for  the 
collection  of  the  HEDIS  Medicare  Measure:  Beta-Blocker  Treatment 
After  a  Heart  Attack.  It  has  found  the  current  information 
processes  to  be  inadequate.  Thus,  the  measure  is  not  reportable 
using  only  administrative  systems.  An  alternative  model  for  the 
data  collection  was  conceptualized  and  assessed  as  vastly 
superior  in  quality  to  the  current  information  model. 

Unfortunately,  this  is  not  a  new  finding  for  the  MHS .  The 
MHS  has  known  that  their  information  management  processes  have 
been  deficient  in  this  area  for  at  least  three  years. 
Regrettably,  for  the  TSP  demonstration  there  is  little  that  can 
be  done  in  the  short-term  to  correct  these  problems. 

This  study  went  to  great  lengths  to  identify  and  review  all 
known  information  systems  utilized  in  the  TSP  demonstration. 
Never  the  less,  it  is  possible,  due  to  the  sheer  number  and  size 
of  the  information  systems  in  the  MHS,  that  the  study  failed  to 
identify  every  potential  information  system  flow  for  the  TSP 
data.  A  single  system  source  for  all  of  the  required  data  is  a 
very  remote  possibility.  If  there  were  such  a  system,  the 
author  would  seriously  question  the  quality  of  its  data  and  the 
quality  of  its  information,  for  all  of  the  aforementioned 


reasons  . 
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The  MHS  is  at  a  critical  juncture.  It  has  spent  millions  of 
dollars  on  information  systems  that  do  not  meet  user 
requirements.  The  GAO  has  already  called  into  question  the  MHS' 
ability  to  effectively  use  its  own  information  systems  to  manage 
healthcare  operations.  If  the  MHS  fails  the  TSP  demonstration, 
specifically  because  of  their  poor  information  systems,  it  is 
possible  that  it  will  loose  much  more  than  just  the  potential 
for  Medicare  reimbursement. 

This  study  proposes  a  relatively  simple  model  that  may  be 
used  to  build  the  types  of  information  systems  (not  just  data 
systems)  that  the  MHS  requires  to  survive.  Some  may  argue  that 
the  conceptual  model  is  a  return  to  the  stove-piped  systems  of 
the  past.  The  author  would  argue  that  this  is  not  the  case  for 
two  reasons.  First,  the  model  is  based  upon  an  identified 
business  process  that  spans  the  breadth  of  the  organization. 
Secondly,  the  model  aligns  information  systems  with  strategic 
initiatives.  Therefore,  it  is  not  like  MEPRS  with  its  focus  on 
resource  management.  Nor  is  it  like  CHCS  with  its  focus  on 
clinical  data.  Further,  it  is  unlike  CEIS  which  touts  itself  as 
a  decision  support  system  but  has  no  user  relevant  focus. 

Another  criticism  may  be  that  the  conceptual  model  presented 
here  is  another  costly  information  system.  The  point  of  this 
criticism  would  be  to  force  the  MHS  to  fix  the  systems  in  which 
millions  of  dollars  have  already  been  invested.  The  fundamental 
flaw  with  this  argument  is  that  it  fails  to  recognize  the 
importance  data  models  have  on  information  quality.  The  very 
foundations  of  the  current  information  systems  are  flawed  (e.g.. 
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there  are  no  common  data  definitions  across  systems) .  The  only 
prudent  alternative  is  to  build  systems  based  upon  a  functional 
data  model.  The  data  model  in  the  alternative  information  model 
is  derived  from  a  proven  TDQM  methodology  using  data  standards 
derived  from  civilian  healthcare  industry  standards.  In 
addition,  all  entities  contributing  to  the  data  flows  are 
required  to  use  the  same  system. 

The  TSP  demonstration  is  under  assault  from  a  number  of 
fronts.  Therefore,  one  may  question  the  decision  to  invest 
additional  resources  toward  rectifying  the  HEDIS  reporting 
problem.  The  response  is  simple.  HEDIS  measurement  is  not  only 
important  to  those  on  the  outside  of  the  organization  (viz.,  the 
GAO  and  HCFA) ,  it  should  also  be  very  important  to  the  leaders 
and  healthcare  providers  within  the  MHS .  HEDIS  measures  are 
designed  to  help  the  healthcare  organization  evaluate  where  they 
stand  in  comparison  to  other  healthcare  organizations .  Every 
responsible  healthcare  leader  must  seek  to  improve  their 
organization.  HEDIS  measurement  provides  an  ideal  way  for  the 
leader  to  do  this.  The  MHS  owes  it  to  every  MTF  commander, 
every  MTF  staff  member,  and  every  MTF  patient  to  accurately 
report  HEDIS  measures. 

It  is  possible  for  the  MHS  to  implement  the  concepts 
outlined  in  the  conceptual  model.  However,  it  requires  strong 
leadership  at  the  highest  level  of  the  organization  to  overcome 
the  inter-service  and  even  intra-service  warfare.  An  analysis 
of  all  the  HEDIS  measures,  in  a  manner  similar  to  that  performed 
in  this  study,  should  be  undertaken.  The  TSP  demonstration 
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provides  fertile  ground  for  such  an  effort  to  begin.  The  TSP 
demonstration  was  and  is  a  major  catalyst  for  changing  business 
practices  within  the  MHS .  If  an  information  model,  such  as  the 
one  presented  here,  were  produced  and  implemented  for  all  HEDIS 
Medicare  measures,  it  would  provide  a  much-needed  foundation  for 
system-wide  improvement. 
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Appendix  1 .  TSP  Enrollment  Data  Flow  Diagram 
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Figure  2-20-N-5  Data  Flow  Charts 

A.  TRICARE  Senior  Option  -  Enrollment  Data  Flow 
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Appendix  2.  TSP  Claims/Clinical  Flow  Diagram 
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Figure  2-20-N-5  Data  Flow  Charts  (Continued) 

B.  TRICARE  Senior  Option  -  Claims /Clinical  Data  Flow 
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